I'm trying to write a script that will grab logs across a network and parse them for relevant information and perform some action (email if there's a critical issue, simply write to a log file if its a warning). I am using an AIX machine with syslogd to process the logs. Right now it is performing like usual, writing all logs to files ... a lot of files.
I was advised to use Perl and Named Pipes to implement the script. I've just spent some time reading up on named pipes and I find them quite fascinating. However, I'm stumped as to how the "flow" of information should work in this situation and how to make perl handle it.
For example, should I create a fifo outside of the script and tell syslogd to write to it by default and have my script on the other end parsing it? Can Perl do that and (for you sysadmins) is this a smart/possible option?
This is my first encounter with Perl and with named pipes.
You can surely create a named pipe in Perl, although it seems to me that for what you are trying to do, it is better to create the named pipe outside of perl, as you are suggesting, and then have syslogd write to it, and read the pipe from perl.
I don't know very well AIX, but this could do for creating a pipe (source):
mkfifo -p /var/adm/syslog.pipe
To have syslogd write to it, define this in /var/adm/syslog.pipe:
*.info |/var/adm/syslog.pipe
Then:
kill -HUP `cat /var/run/syslogd.pid`
You could also put all this stuff into your perl script: in case the pipe did not exist or syslogd were not using it, the script would arrange all required things for you.
Possibly you could provide some more details as to what you are trying to do, if you need more help.
Related
I'm working with an existing framework of WinDbg scripts that go through a series of test scripts Test1.txt, Test2.txt, etc., which are generated by C++ code and which output results.
For example a chunk of one of the test scripts would be,
.if (($spat(#"${var}","18300.000000")==1))
{
.logappend C:\Tests\TestResults.txt
.printf "TestNumber=\t1\tExpected=\t18300.000000\tActual=\t%.6f\t******PASSED******\n",poi(poi(#$t2+#$t6)+0x10)
.logclose
}
I'm trying to add functionality that will create a file whose name displays the current # of the test being run, so that users can see their progress without needing to open a file.
My thought process was that I would set up the script generator, so that at the start of Test #N, it would add a line to the script to create a file 'currentlyRunningTestN.txt', and at the end of Test #N, it would add a line to the script to delete that file. However, I don't see any delete function in the WinDbg meta command glossary: https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/meta-commands, or in the list of supported C functions like printf. Am I just missing something, or is deleting files not supported by WinDbg (or equivalently renaming files, which would also serve my purpose?) If deleting/renaming don't work, is there another way to achieve the functionality I'm looking for?
With the .shell command, you can execute any DOS-like command. Although I never tried deleting a file, it should be possible.
As you may have noticed, WinDbg scripting does not always work on first attempt, please make sure your scripting will not result in a big data loss on your customer's PC whilst deleting files.
From my PERL script, I am calling child shell script.
There are few db environment variables which are exported by child shell script
But when I try to use those in perl script, they are not shown. Here is my code:
my $commanLine = ". SetConnection.sh -n $TaskName";
system $commanLine;
my $dbConnectString = "$ENV{'DB_USER'}/$ENV{'DB_PASSWORD'}";
print "$dbConnectString";
Please suggest.
TL;DR
Exported variables are inherited by child processes from the parent. You can't modify the environment of the parent process from the child directly, but you can certainly exchange data using files, pipes, or other forms of interprocess communication.
Source a Perl File Holding Variables
The easiest solution is to have the child process write a file that can then be sourced by the parent. For example, security issues aside, SetConnection.sh could write to a file like /tmp/variables.pl, which you could then source as a Perl script inside the parent script.
For example, consider the following file, presumably written by the child process:
# /tmp/foo.pl
$foo='bar';
Now you require the file in your parent script:
$ perl -e 'require "/tmp/foo.pl"; print "$foo\n"'
bar
This isn't really very secure, but it does work. Think of it as similar to eval, along with race conditions and access issues. Nevertheless, it's a very pragmatic solution.
Use a Real Configuration File
Alternatively, you could use a format like JSON, YAML, or CSV (created any way you like, including by your child process) to create a configuration file which you could then parse for values. This is generally the best approach, but your use case may vary.
The benefit of this approach is that you can validate and sanitize values, and don't need to worry about the security or uniqueness of temp files. It's really the right way to do these things, but will require much more work on your part.
I have a named pipe server in my software, which I have accessed using C# and Python. I have a customer asking me if it's possible to access the named pipe through Simulink, but I have never used that software. Google and Stackoverflow don't seem to contain any examples of this, but I'm not sure that means that it's not possible. Does anyone know for sure whether Simulink is or isn't capable of accessing the named pipe server in another program?
I don't know what simulink is, but a named pipe usually just shows up like a file... they would have to go to some length to detect that a file was a pipe (fstat/lstat)
but you can make one to test with like...
mkfifo dog
echo "bark"
then try to open that file in simulink...
that is just basically the semantics of opening a file, if simulink tries to seek around in the file ... it will
fail...
you should read about what a fifo is, and play with it, like in the above example:
try in another shell: cat dog...
Is it possible to execute a Perl script within a Stata .do file?
I have a Stata .do file in which I make some manipulations to the data set and arrange it in a certain way. Then I have a Perl script in which I take one of the variables at this point, apply a Perl package to it, and make a transformation to one of the variables. In particular, I use Perl's NYSIIS function, resulting in a very short script. After this output is provided in Perl, I'd like to continue with some additional work in Stata.
Two alternatives that come to mind but that are less desirable are:
Write Stata code to do nysiis but I prefer to use Perl's built-in function.
outsheet and save the output from the Stata .do file as a .txt for Perl. Then do the Perl script separately to get another .txt. Then read in that .txt to Stata to a new .do file and resume.
Your approach number 2 is what I use to call other programs to operate on Stata data. As Nick says, Stata won't necessarily wait for your output, unless you ask it to. You first outsheet the text file, then call the Perl script from Stata using ! to run something on the command line. Finally, have Stata periodically check for the result file, using a while loop and the sleep command so Stata isn't constantly checking.
outsheet using "perl_input.txt"
!perl my_perl_script.pl
while (1) {
capture insheet using "perl_output.txt", clear
if _rc == 0 continue, break
sleep 10000
}
!rm perl_output.txt
Here, your formatted data is saved from Stata as perl_input.txt. Next, your Perl script is run from the command line, and using a while loop, Stata checks for the output every 10 seconds (sleep takes arguments in milliseconds). When it finds the output file, it breaks out of the while loop. The last line is a good idea so that when you re-use the code you don't run the risk of using the Perl output from the last run.
I think the main issue is that although you can use the shell to call up something else, Stata is not going to wait for the results.
Start with help shell to see what is possible, but your #2 does sound easiest.
I have a quick question about creating files with perl and executing them. I wanted to know if it was possible to generate a file using perl (I actually need a .bat script) and then execute this file internally to the program. I know I can create files, and I have with perl, however, I'm wanting to do this internally to the program. So, what I want it to do is actually create a batch script internally to the program (no file is actually written to the disk, everything remains in memory, or the perl program), and then once it completes the writing of the file, I'd like to be able to actually execute this file, and then discard the file it just wrote. I'm basically trying to have it create a batch script on the fly, so that I can just have output text files from the output of the script, rather than creating the batch script on disk, then executing it, and then deleting the batch file from disk when its done.
Can this be done and how would I go about doing this?
Regards,
Drew
Do you really need a batch script? Perhaps everything you want to do can be done directly from Perl or invoked directly by Perl via its system command.
If a batch script is essential, what's wrong with creating a temporary file for the script and then executing it with system? See File::Temp, which will even delete the temporary file automatically after you are done.
If the virtual-batch-file strategy is unavoidable, you might be able to leverage the /C and maybe /S options of cmd. Something like this:
use strict;
use warnings;
my #batch_commands = (
'dir',
q{echo "Make sure quoting isn't busted"},
'ipconfig',
);
# Use & or &&, depending on your needs. Run `cmd /?` for details.
my $virtual_bat_file = join " &\n", #batch_commands;
system "cmd /C $virtual_bat_file";
But this feels very wrong. There has to be a better way to accomplish whatever the larger goal of your application is. By the way, when you run cmd /? to learn about /C, /S, and & vs. &&, you'll quickly appreciate how terrible it is in the Land of Batch. Stay away if at all possible.
open the file; create the contents; close the file; execute the file (with system(), for example); remove the file.