I'm trying to develop a software featuring multithreading.
On Linux the script works fine, the module i've used is ''forks''.
In other words, no problem of shared handlers between threads while querying db and stuff like that.
Once i've been trying to run the script on windows (Strawberry Perl), when i try to cpan install forks it says that ''forks module'' is not supported for the current version of my OS (64bit).
Moving over, i decided to use ''threads'' instead, but i gained the following error, almost certainly linked with shared handlers between threads.
''
Thread 1 terminated abnormally: DBD::SQLite::db prepare failed: handle 2 is owned by thread d97fe8 not current thread 3a01058 (handles can't be shared between threads and your driver may need a CLONE method added) at file.pl line 180, line 1.
''
How to fix the aforementioned problem and make the script runnable on windows Strawberry Perl?
In general each thread or process needs to have its own handle to the database. Create a new handle after each fork or within each threads::create.
First of all, thanks everybody for the prompt answer and the very good suggestions.
That's my solution.
Apparently, it seems there's no need to create a new db handler for each thread, just connect to the database each and every time you create a new thread.
So, if you're using ''threads'', i think it should be enough to include the following line within the ''threads->create'' method to make it runnable:
$dbh = DBI->connect( "dbi:SQLite:db_name.dbl" ) || die "Cannot connect: $DBI::errstr";
As you can see, the handler is static but it's not literally ''shared'' between
threads, as it is created each and every time you connect to the db, so each and every time you run a new thread!
Please, don't ask me why the original script works fine in Linux using ''forks'' and without this trick...it seems that with ''forks'' you don't need to care about shared handlers.
Thanks again
Related
I have a proc which spawns a child process on port 2600. it connects to the process with handle 6, extracts some data, kills the process and then it starts another child process on the same port, I connect to it and when I try to run my commands on the handle I get this following error:
'Cannot write to handle 6. OS reports: Bad File Descriptor
The process is running and I can connect to it manually and get my data. I have a table tracking the connection and I can see the handle for the second time the child process is spawned is also 6. Any suggestions on why this is happening?
UPDATE: Just to make it clear. When I close the handle I send an async message to the proc with "exit 1". Could this be that the process is not killed before I open a connection to it, gets killed after that and then what I think it's the new process is just a garrbage handle to the old, now defunct process?
UPDATE: I do flush the handle after I send "exit 1" and the new process seem to start ok. I can see the new process running on the process (they have different names)
UPDATE: Even thought connection is successful the handle is not added to .z.W. so the handle doesn't appear in .z.W
UPDATE: found the issue. the .IPC.SCON function which I was using which i thought was just a simple wrapper for an error trap ... has logic to check a table of cached handles for the same hst:prt and take that one instead of opening another one. because my process was running on the same host:port, it was using the old cached handle instead of opening it again. thank you all for your help.
What version of kdb are you using? Can you run the below without issues?
KDB+ 4.0 2020.08.28 Copyright (C) 1993-2020 Kx Systems
q)system"q -p 5000";system"sleep 1";h:hopen 5000;0N!h".z.i";neg[h](exit;0)
4475i
q)system"q -p 5000";system"sleep 1";h:hopen 5000;0N!h".z.i";neg[h](exit;0)
4481i
q)system"q -p 5000";system"sleep 1";h:hopen 5000;0N!h".z.i";neg[h](exit;0)
4487i
Update: Without seeing your code or the order of execution, it's hard to pinpoint what the issue is. If the process wasn't terminated before you attempt to start the new process, you would see an Address already in use error. If the process isn't up when you attempt to connect you would see a 'hop error which means hopen failed.
Also, async messages are not sent immediately so depending on the execution order of your code, you may need to flush the async 'exit' message, but like I mentioned, if the original child process is still up when you attempt to start another, you would get an address clash
Update 2: As per my comment above
q)system"q -p 5000"; system"sleep 1"; h:#[hopen;(`::5000;2000);0Ni]; 0N!h".z.i"; neg[h](exit;1)
4365i
q)system"q busy.q -p 5000"; system"sleep 1"; #[hopen;(`::5000;2000);0Ni]; h".z.i" // hopen times out + handle variable not amended
'Cannot write to handle 4. OS reports: Bad file descriptor
[0] system"q busy.q -p 5000"; system"sleep 1"; #[hopen;(`::5000;2000);0Ni]; h".z.i" // hopen times out + handle variable not amended
^
q).z.W
q)
I think what you have written in your UPDATE is correct. The process is trying to connect to that port before your new process is running on it.
I think the best option would be for the newly started process to initiate the connection to the parent rather, that way you know it will be running and don't have to introduce any sleeps.
Other option is to try and reconnect on a timer, once a successful connection occurs remove it from the timer and continue with what you are trying to do.
The problem statement in short: Is there a way in LSF to pass a signal SIGCONT/SIGTSTP to all processes running within a job?
I have a Perl wrapper script that runs on LSF (Version 9.1.2) and starts a tool (Source not available) on the same LSF machine as the Perl script.
The tool starts 2 processes, one for license management and another for doing the actual work. It also supports an option where sending SIGSTSP/SIGCONT to both processes will release/reacquire the license (which is what I wish to achieve).
Running bkill -s SIGCONT <JOB_ID> only resumes the tool process and not the license process, which is a problem.
I tried to see if I can send the signals to the Perl script's own PGID, but the license process starts its own process group.
Any suggestions to move forward through Perl or LSF options are welcome.
Thanks,
Abhishek
I tried to see if I can send the signals to the Perl script's own PGID, but the license process starts its own process group.
This is likely your problem right here. LSF keeps track of "processes running within the job" by process group. If your job spawns a process that runs within its own process group (say by daemonizing itself) then it essentially is a runaway process out of LSF's control -- it becomes your job's responsibility to manage it.
For reference, see the section on "Detached processes" here.
As for options:
I think the cgroups tracking functionality helps in a lot of these cases, you can ask your admin if LSF_PROCESS_TRACKING and LSF_LINUX_CGROUP_ACCT are set in lsf.conf. If they aren't, then you can ask him to set them and see if that helps for your case (you need to make sure the host you're running on supports cgroups). In 9.1.2 this feature is turned on at installation time, so this option might not actually help you for various reasons (your hosts don't have cgroups enabled for example).
Manage the license process yourself. If you can find out the PID/PGID of the license process from within your perl script, you can install custom signal handlers for SIGCONT/SIGSTP in your script using sigtrap or the like and forward them to the license process yourself when your script receives them through bkill. See here.
I have an example and question regarding unix/apache session scope. Here is the test script I am using:
#! /usr/bin/perl -I/gcne/etc
$pid = $$;
system("mkdir -p /gcne/var/nick/hello.$pid");
chdir "/gcne/var/nick/hello.$pid";
$num = 3;
while($num--){
system("> blah.$pid.$num");
#sleep(5);
system("sleep 5");
}
system("> blahDONE.$pid");
I have noticed that if I call this script TWICE from a web browser that it will execute these requests in sequence — a total of 30 seconds. How does Perl/unix deal with parallel execution and using system commands? Is there a possibility that I get cross-session problems when using system calls? Or does apache treat each of these server calls as a new console session process?
In this example, I'm basically trying to test whether or not different PID files would be created in the "wrong" PID folder.
CentOS release 5.3
Apache/2.2.3 Jul 14 2009
Thanks
If you call the script via the normal CGI interface, then each time you request a web page your script is called. This means each time it gets a new process ID. Basically for CGI's the interface between Apache and your programm are the commandline args, the environment variables and STDOUT and STDERR. Otherwise everything is a normal command call.
Situation is a little different when you use mechanism like mod_perl, but it seems you don't do this ATM.
Apache does not do any synchronisation, so you can expect up to MaxClients (see apache docs) parallel calls of your script.
P.S. The environment variables between a call from apache and from shell are a bit different, but this is not relevant for your question (but you'll probably wonder if e.g. USER or similar variables are missing).
See also for more information: http://httpd.apache.org/docs/2.4/howto/cgi.html
Especially: http://httpd.apache.org/docs/2.4/howto/cgi.html#behindscenes
A browser may only issue one call at a time (tested with firefox), so when testing it may appear requests are handled one after another. This is not server related, but caused by the web browser.
I've searched and found several very similar questions to mine but nothing in those answers have worked for me yet.
I have a perl CGI script that accepts a file upload. It looks at the file and determines how it should be processed and then calls a second non-CGI script to do the actual processing. At least, that's how it should work.
This is running on Windows with Apache 2.0.59 and ActiveState Perl 5.8.8. The file uploading part works fine but I can't seem to get the upload.cgi script to run the second script that does the actual processing. The second script doesn't communicate in any way with the user that sent the file (other than it sends an email when it's done). I want the CGI script to run the second script (in a separate process) and then 'go away'.
So far I've tried exec, system (passing a 1 as the first parameter), system (without using 1 as first parameter and calling 'start'), and Win32::Process. Using system with 1 as the first parameter gave me errors in the Apache log:
'1' is not recognized as an internal or external command,\r, referer: http://my.server.com/cgi-bin/upload.cgi
Nothing else has given me any errors but they just don't seem to work. The second script logs a message to the Windows event log as one of the first things it does. No log entry is being created.
It works fine on my local machine under Omni webserver but not on the actual server machine running Apache. Is there an Apache config that could be affecting this? The upload.cgi script resides in the d:\wwwroot\test\cgi-bin dir but the other script is elsewhere on the same machine (d:\wwwroot\scripts).
There may be a security related problem, but it should be apparent in the logs.
This won't exactly answer your question but it may give you other implementation ideas where you will not face with potential security and performance problems.
I don't quite like mixing my web server environment with system() calls. Instead, I create an application server (with POE usually) which accepts the relevant parameters from the web server, processes the job, and notifies the web server upon completion. (well, the notification part may not be straightforward but that's another topic.)
I am a newbie to perl. I am using perl expect module to spawn to a remote system. Execute a set of commands there one after another using the send module(like $exp->send("my command as string goes here\n"). The problem is the commands that I execute take some time for processing . And before all the command finish the remote machine gets timed out and I come back to my host machine prompt. Can you please help me how to handle this.?
I have one more question. I have a command which returns 2 values after execution(say I am doing a print for 2 values on remote machine). I want to capture these 2 values and pass as argument to the next command using send module. How do I do this.
Pls help me with this problem.
Thanks.
I just found out something about the expect module. There is an undef option that can be used with expect like $exp->expect(undef). This will wait indefinitely and lets all commands finish their processing. But the problem is that, it does not return back the control to the host machine. There is one more option of using expect with eof which will wait until it encounters an eof and then returns to the host machine. Although no idea precisely how to use it. An elegant solution that I found is to use ssh to run commands on remote machine rather than using expect in which case we do not have to deal with timeouts. :)
I just found out something about the expect module. There is an undef option that can be used with expect like $exp->expect(undef). This will wait indefinitely and lets all commands finish their processing. But the problem is that, it does not return back the control to the host machine. There is one more option of using expect with eof which will wait until it encounters an eof and then returns to the host machine. Although no idea precisely how to use it. An elegant solution that I found is to use ssh to run commands on remote machine rather than using expect in which case we do not have to deal with timeouts. :)