In linux, to concatenate all files under a folder, you can do file=FOLDER/*; cat $file > ONEFILE, I also want to use this in my perl script so I coded like system("file=$folder/*");
system("cat \$file > $out");
But it won't work when I run the perl program, the $out was assigned a file name as my $out = "outfile";. The outfile always keeps at 0 bit. What's wrong here.
The first line sets the $file environment variable in a new shell process:
system "file=$folder/*";
The second line starts a new shell process with a new environment:
system "cat \$file > $out";
Since it's a new process, with a new environment, your previous $file variable is no longer set, so you are really running the following shell command:
cat > $out
Do this instead:
system "cat '$folder/'* > '$out';
Note - I also added quotes, which will help if your paths may contain spaces. However, it's still not safe against all forms of input, so don't pass any user input to that command without validating it first.
What's about exec in Perl?
perl -e 'exec "cat *.txt"'
Related
Below is the code I'm trying to execute. I have mentioned the line 266 in the code. I have added that code to remove the blank lines in the log file. I'm not sure whether we can run the perl command inside a Perl script. Is there another way that I can run this so that I can remove the blank lines in the log file?
Below is the error I'm getting while running through the Perl script:
syntax error at ./reportJBossErrors.pl line 266, near "n -e "
Execution of ./reportJBossErrors.pl aborted due to compilation errors.
Here is a portion of the code, showing line 266:
sub main {
readConfiguration($config_file);
$short_hostname = `hostname | cut -f 1 -d.`;
chomp $short_hostname;
getFileandInstance($short_hostname);
$yesterday = getYesterday();
validateEnvironment();
$log_file = getLogFile($FMASK,$yesterday);
perl -i -n -e "print if /\S/" $log_file; # 266 line. This is where I'm getting the compilation error
processFile($log_file);
$html_out = writeEmail();
sendEmail($CONFIG{"FROMADDR"},$CONFIG{"TOADDR"},"Normal",
"JBOSS",$short_hostname,$log_file,$CONFIG{ENVTYPE},$html_out);
}
You can not call the perl command inside a Perl program as if it were a Perl builtin function. You can use system to run an external command:
my $cmd = 'perl -i -n -e "print if /\S/"';
system "$cmd $log_file";
You need to be careful of quoting. Since you have a file name/path in the Perl variable $logfile, which you want to interpolate, that can go inside double quotes. Since you do not want to interpolate \S, that should go in single quotes.
You cannot invoke the perl executable inside a Perl program as if it were a Perl builtin function. Instead, use the list form of system to run an external command. Don't forget to check if the command succeeded:
my #cmd = (perl => '-i', '-n', '-e', 'print if /\S/', $log_file);
system(#cmd) == 0
or die "system #cmd failed: $?";
In general, I would recommend using the full path to perl rather than relying on $PATH.
Also, if you need to keep track of status etc, use Capture::Tiny to get both STDOUT and STDERR of the command you are running so that you can log error information.
Below command in perl script is failing if there are many file in #file_to_tar, but it is working fine if we have less files in the array (#file_to_tar).
my $tar_command = "cd $ProcVars->{dropbox_dir}; tar -cvzf SmartMiles.$ProcVars->{batch_nb}.tar.gz -P #file_to_tar --remove-files";
Can some one please help me to fix the issue.
The best way is pass a very long list of files to tar is using the --from-file option:
tar -czf myarchive.tar.gz -P --from-file=$filelist --remove-files
You can also make it read the list of files from standard input by using --from-file=-
See https://www.gnu.org/software/tar/manual/html_node/files.html for more information.
You might consider, rather than calling a shell command, using the Archive::Tar Perl module instead. In particular, the "create_archive" method. This also would sidestep any potential problems arising from special characters or whitespace in the file names being interpreted by the shell your Perl code is invoking.
it is working now.
my $file_location = "$ProcVars->{dropbox_dir}/Archive_Files.csv"; open(DATA, ">$file_location") or die $!; foreach $a (#file_to_tar) { print DATA "$a\n"; } close DATA;
my $tar_command = "cd $ProcVars->{\dropbox_dir};tar -czf ABC.$ProcVars->{batch_nb}.tar.gz -P --files-from $file_location --remove-files";
RHEL6
I have a c-shell script that runs a perl script. After dumping tons of stuff to stdout, it determines where (what dir) the parent shell should cd to when the perl script finishes. But that's a string, not an int which is all I can pass back with "exit()".
Storing the name of the dir in a file which the c-shell script can read is what I have now. It works, but is not elegant. Is there a better way to do this ? Maybe a little chunk of memory that I can share with the perl script ?
Short:
Redirect Perl's streams and restore in the end to print that info, taken by the shell script
Or, print that last and the shell script can pass output to the console and take the last line
Or, use a named pipe (either shell) or specific file descriptors (not csh) for that print
When the Perl script prints out that name you can assign it to a variable
in the shell script
#!/bin/csh
set DIR `perl -e'print "dir_name"'`
while in bash
#!/bin/bash
DIR="$(perl -e'print "dir_name"')"
where $(...) is preferred for the command substitution.
But those other prints to console from the Perl script then need be handled
One way is to redirect all output in Perl script other than that one print, what can be controlled by a command-line option (filename to which to redirect, which shell script can print out)
Or, take all Perl's output and pass it to console, the last line being the needed "return." This puts the burden on the Perl script to print that last (perhaps in an END block). The program's output can be printed from the shell script after it completes or line by line as it is emitted.
Or, use a named pipe (both shells) or a specific file descriptor (bash only) to which the Perl script can print that information. In this case its streams go straight to the console.
The question explicitly mentions csh so it is given below. But I must repeat the old and worn fact that shell scripting is far better done in bash than in csh. I strongly recommend to reconsider.
bash
If you need the program's output on the console as it goes, take and print it line by line
#!/bin/bash
while read line; do
echo "$line"
DIR=$line
done < <(perl script.pl)
echo "$DIR"
Or, if you don't need output on the console before the script is finished
#!/bin/bash
mapfile -t lines < <(perl script.pl)
DIR="${lines[-1]}"
printf '%s\n' "${lines[#]}" # print script.pl's output
Or, use file descriptors for that particular print
F=$(mktemp) # safe filename
exec 3> "$F" # open fd 3 to write to it
exec 4< "$F" # open fd 4 to read from it
rm -f "$F" # remove file(name) for safety; opened fd's can still access
perl -E'$fd=shift; say "...normal prints to STDOUT...";
open(FH, ">&=$fd") or die $!;
say FH "dirname";
close FH
' 3
read dir_name <&4
exec 3>&- # close them
exec 4<&-
echo "$dir_name"
I couldn't get it to work with a single file descriptor for both reading and writing (exec 3<> ...), I think because the read can't rewind after the write, thus separate descriptors are used.
With a Perl script (and not the demo one-liner above) pass the fd number as a command-line option. The script can then do this only if it's invoked with that option.
Or, use a named pipe very similarly to how it's done for csh below. This is probably best here, if the manipulation of the program's STDOUT isn't to your liking.
csh
Iterate over the program's (completed) output line by line
#!/bin/csh
foreach line ( "`perl script.pl`" )
echo "$line"
set dir_name = "$line"
end
echo "Directory name: $dir_name"
or extract the last line first and then print the whole output
#!/bin/csh
set lines = ( "`perl script.pl`" )
set dir_name = $lines[$#]
# Print program's output
while ( $#lines )
echo "$lines[1]"
shift lines
end
or use a named pipe
set fifo_name = "/tmp/fifo$$" # or use mktemp
mkfifo "$fifo_name"
( perl script.pl --fifo $fifo_name [other args] & )
set dir_name = `cat "$fifo_name"`
rm -f $fifo_name
echo "dir name from FIFO: $dir_name"
The Perl command is in the background since FIFO blocks until written and read. So if the shell script were to wait for perl ... to complete the Perl script would block as it's writing to FIFO (since that's not being read) so shell would never get to read it; we would deadlock. It is also in a subshell, with ( ), so to avoid the informational prints about the background job.
The --fifo NAME command-line option is needed so that Perl script knows what special file to use (and not to do this if the option is not there).
For an in-line example replace ( perl script ...) with this one-liner, used above as well
( perl -E'$ff = shift; say qq(\t...normal prints to STDOUT...);
open FF, ">$ff" or die $!;
say FF "dir_name_$$";
close FF
' $fifo_name
& )
(broken over lines for readability)
I'm very new to Perl, and I would like to make a program that creates a directory and moves a file into that directory using the Unix command like:
mkdir test
Which I know would make a directory called "test". From there I would like to give more options like:
mv *.jpg test
That would move all .jpg files into my new directory.
So far I have this:
#!/usr/bin/perl
print "Folder Name:";
$fileName = <STDIN>;
chomp($fileType);
$result=`mkdir $fileName`;
print"Your folder was created \n";
Can anyone help me out with this?
Try doing this :
#!/usr/bin/perl
use strict; use warnings;
print "Folder Name:";
$dirName = <STDIN>;
chomp($dirName);
mkdir($dirName) && print "Your folder was created \n";
rename $_, "$dirName/$_" for <*.jpg>;
You will have a better control when using built-in perl functions than using Unix commands. That's the point of my snippet.
Most (if not all) Unix commands have a corresponding version as a function
e.g
mkdir - see here
mv - See here
Etc. either get a print out of the various manual pages (or probably have a trip down to the book shop - O'Reilly nut shell book is quite good along with others).
In perl you can use bash commands in backticks. However, what happens when the directory isn't created by the mkdir command? Your program doesn't get notified of this and continues on its merry way thinking that everything is fine.
You should use built in command in perl that do the same thing.
http://perldoc.perl.org/functions/mkdir.html
http://perldoc.perl.org/functions/rename.html
It is much easier to trap errors with those functions and fail gracefully. In addition, they run faster because you don't have to fork a new process for each command you run.
Perl has some functions similar to those of the shell. You can just use
mkdir $filename;
You can use backquotes to run a shell command, but it is only usefull if the command returns anything to its standard output, which mkdir does not. For commands without output, use system:
0 == system "mv *.jpg $folder" or die "Cannot move: $?";
As I understand (Perl is new to me) Perl can be used to script against a Unix command line. What I want to do is run (hardcoded) command line calls, and search the output of these calls for RegEx matches. Is there a way to do this simply in Perl? How?
EDIT: Sequence here is:
-Call another program.
-Run a regex against its output.
my $command = "ls -l /";
my #output = `$command`;
for (#output) {
print if /^d/;
}
The qx// quasi-quoting operator (for which backticks are a shortcut) is stolen from shell syntax: run the string as a command in a new shell, and return its output (as a string or a list, depending on context). See perlop for details.
You can also open a pipe:
open my $pipe, "$command |";
while (<$pipe>) {
# do stuff
}
close $pipe;
This allows you to (a) avoid gathering the entire command's output into memory at once, and (b) gives you finer control over running the command. For example, you can avoid having the command be parsed by the shell:
open my $pipe, '-|', #command, '< single argument not mangled by shell >';
See perlipc for more details on that.
You might be able to get away without Perl, as others have mentioned. However, if there is some Perl feature you need, such as extended regex features or additional text manipulation, you can pipe your output to perl then do what you need. Perl's -e switch let's you specify the Perl program on the command line:
command | perl -ne 'print if /.../'
There are several other switches you can pass to perl to make it very powerful on the command line. These are documented in perlrun. Also check out some of the articles in Randal Schwartz's Unix Review column, especially his first article for them. You can also google for Perl one liners to find lots of examples.
Do you need Perl at all? How about
command -I use | grep "myregexp" && dosomething
right in the shell?
#!/usr/bin/perl
sub my_action() {
print "Implement some action here\n";
}
open PROG, "/path/to/your/command|" or die $!;
while (<PROG>) {
/your_regexp_here/ and my_action();
print $_;
}
close PROG;
This will scan output from your command, match regexps and do some action (which now is printing the line)
In Perl you can use backticks to execute commands on the shell. Here is a document on using backticks. I'm not sure about how to capture the output, but I'm sure there's more than a way to do it.
You indeed use a one-liner in a case like this. I recently coded up one that I use, among other ways, to produce output which lists the directory structure present in a .zip archive (one dir entry per line). So using that output as an example of command output that we'd like to filter, we could put a pipe in and then use perl with the -n -e flags to filter the incoming data (and/or do other things with it):
[command_producing_text_output] | perl -MFile::Path -n -e \
"BEGIN{#PTM=()} if (m{^perl/(bin|lib(?!/site))}) {chomp;push #PTM,$_}" ^
-e "END{#WDD=mkpath (\#PTM,1);" ^
-e "printf qq/Created %u dirs to reflect part of structure present in the .ZIP file\n/, scalar(#WDD);}"
the shell syntax used, including: quoting of perl code and escaping of newlines, reflects CMD.exe usage in Windows NT-like consoles. If you need to, mentally replace
"^" with "\" and " with ' in the appropriate places.
The one-liner above adds only the directory names that start with "perl/bin" or
"perl/lib (not followed by "/site"); it then creates those directories. You wind
up with a (empty) tree that you can use for whatever evil purposes you desire.
The main point is to illustrate that there are flags available (-n, -p) to
allow perl to loop over each input record (line), and that what you can do is unlimited in terms of complexity.