I have a bash command, get-modified-perl-files, that returns all the Perl files I have modified in my repository. I would like to use perltidy on all of these files.
I created a bash function to do the job:
tidy() {
for f in `get-modified-perl-files`
do
echo $f
perltidy -b $f
done
}
According to the help page of perltidy, the -b option should create a backup of my original file and modify it in-place:
-b backup original to .bak and modify file in-place
However, when I launch my bash function, no backup is created. My files are not modified, but the output of perltidy is printed on the standard output. As a consequence, I decided to change my call to perltidy that way:
\cp $f $f.bak
perltidy $f > $f
Now, when I run my command, the backup of my file is correctly done, but the original file is emptied, and the following message is displayed:
skipping file: file.pl: Zero size
I've found a workaround which gives the result I want, but it seems far-fetched:
\cp -f $f $f.bak
echo "$(perltidy $f)" > $f
Why the -b option doesn't work? Is there a way to do the same job without using this weird redirection?
EDIT: Here is my .perltidyrc file:
--perl-best-practices
--no-standard-error-output
--closing-side-comments
--closing-side-comment-interval=10
--blanks-before-subs
--blanks-before-blocks
--maximum-line-length=130
By default perltidy does not print the file contents to STDOUT. To do so requires the -st option (or --standard-output). Since you are not using this option on the perltidy command line, there is likely a .perltidyrc file with -st in it that is being used.
To ignore the .perltidyrc file, use the -npro (--noprofile) option:
perltidy -npro -b $f
Refer to the "Using a .perltidyrc command file" section of the man page for your installed version:
perldoc perltidy
For addition debug information, you can run:
perltidy -dpro
perltidy -dop
Another possibility is that you aliased the perltidy command to perltidy -st. You should be able to avoid an alias with:
\perltidy -npro -b $f
Now that you edited your Question to show your .perltidyrc file, it looks like the culprit is:
--perl-best-practices
Either change the rc file, or ignore it as above.
See also Perltidy always prints to standard out
perltidy $f > $f
This will never do what you want, with any program. When you run a program with > $f, that tells the shell that you want the program to run with its stdout connected to $f. So before the program is run, the shell opens $f for writing, which destroys the contents of the file. Then it connects the handle to stdout in the child, then it runs perltidy, which tries to read $f and finds... nothing, because the original contents were already wiped out. Not a recipe for success. This is why perltidy has its own "in-place editing" feature in the first place.
Related
RHEL6
I have a c-shell script that runs a perl script. After dumping tons of stuff to stdout, it determines where (what dir) the parent shell should cd to when the perl script finishes. But that's a string, not an int which is all I can pass back with "exit()".
Storing the name of the dir in a file which the c-shell script can read is what I have now. It works, but is not elegant. Is there a better way to do this ? Maybe a little chunk of memory that I can share with the perl script ?
Short:
Redirect Perl's streams and restore in the end to print that info, taken by the shell script
Or, print that last and the shell script can pass output to the console and take the last line
Or, use a named pipe (either shell) or specific file descriptors (not csh) for that print
When the Perl script prints out that name you can assign it to a variable
in the shell script
#!/bin/csh
set DIR `perl -e'print "dir_name"'`
while in bash
#!/bin/bash
DIR="$(perl -e'print "dir_name"')"
where $(...) is preferred for the command substitution.
But those other prints to console from the Perl script then need be handled
One way is to redirect all output in Perl script other than that one print, what can be controlled by a command-line option (filename to which to redirect, which shell script can print out)
Or, take all Perl's output and pass it to console, the last line being the needed "return." This puts the burden on the Perl script to print that last (perhaps in an END block). The program's output can be printed from the shell script after it completes or line by line as it is emitted.
Or, use a named pipe (both shells) or a specific file descriptor (bash only) to which the Perl script can print that information. In this case its streams go straight to the console.
The question explicitly mentions csh so it is given below. But I must repeat the old and worn fact that shell scripting is far better done in bash than in csh. I strongly recommend to reconsider.
bash
If you need the program's output on the console as it goes, take and print it line by line
#!/bin/bash
while read line; do
echo "$line"
DIR=$line
done < <(perl script.pl)
echo "$DIR"
Or, if you don't need output on the console before the script is finished
#!/bin/bash
mapfile -t lines < <(perl script.pl)
DIR="${lines[-1]}"
printf '%s\n' "${lines[#]}" # print script.pl's output
Or, use file descriptors for that particular print
F=$(mktemp) # safe filename
exec 3> "$F" # open fd 3 to write to it
exec 4< "$F" # open fd 4 to read from it
rm -f "$F" # remove file(name) for safety; opened fd's can still access
perl -E'$fd=shift; say "...normal prints to STDOUT...";
open(FH, ">&=$fd") or die $!;
say FH "dirname";
close FH
' 3
read dir_name <&4
exec 3>&- # close them
exec 4<&-
echo "$dir_name"
I couldn't get it to work with a single file descriptor for both reading and writing (exec 3<> ...), I think because the read can't rewind after the write, thus separate descriptors are used.
With a Perl script (and not the demo one-liner above) pass the fd number as a command-line option. The script can then do this only if it's invoked with that option.
Or, use a named pipe very similarly to how it's done for csh below. This is probably best here, if the manipulation of the program's STDOUT isn't to your liking.
csh
Iterate over the program's (completed) output line by line
#!/bin/csh
foreach line ( "`perl script.pl`" )
echo "$line"
set dir_name = "$line"
end
echo "Directory name: $dir_name"
or extract the last line first and then print the whole output
#!/bin/csh
set lines = ( "`perl script.pl`" )
set dir_name = $lines[$#]
# Print program's output
while ( $#lines )
echo "$lines[1]"
shift lines
end
or use a named pipe
set fifo_name = "/tmp/fifo$$" # or use mktemp
mkfifo "$fifo_name"
( perl script.pl --fifo $fifo_name [other args] & )
set dir_name = `cat "$fifo_name"`
rm -f $fifo_name
echo "dir name from FIFO: $dir_name"
The Perl command is in the background since FIFO blocks until written and read. So if the shell script were to wait for perl ... to complete the Perl script would block as it's writing to FIFO (since that's not being read) so shell would never get to read it; we would deadlock. It is also in a subshell, with ( ), so to avoid the informational prints about the background job.
The --fifo NAME command-line option is needed so that Perl script knows what special file to use (and not to do this if the option is not there).
For an in-line example replace ( perl script ...) with this one-liner, used above as well
( perl -E'$ff = shift; say qq(\t...normal prints to STDOUT...);
open FF, ">$ff" or die $!;
say FF "dir_name_$$";
close FF
' $fifo_name
& )
(broken over lines for readability)
I am trying to code a Perl subroutine that returns an array of files that has been modified and submitted to the Perforce repository from $previous_date until now. This is how the subroutine looks like:
sub p4_files {
my ($previous_date) = #_;
my $files = "//depot/project/design/...rtl.sv"
my $p4cmd = "p4 files -e $files\#$previous_date,\#now";
my #filelist = `$p4cmd`;
chomp #filelist;
return #filelist;
}
The subroutine works as expected if there are files submitted between the given dates. However, it happens that no new changes are made, and executing the p4 files command returns a message instead:
prompt% p4 files -e //depot/project/design/...rtl.sv\#25/05/2017,\#now
prompt% //depot/project/design/...rtl.sv\#25/05/2017,\#now - no revision(s) after that date.
How should I handle this in my Perl script? I would like to exit the script when such a situation is encountered.
Unfortunately, p4 returns exit code 0 regardless of whether it finds some files or whether it returns the "no revision(s) after that date" message. That means you have to parse the output.
The simplest solution is probably to exit the script if $filelist[0] =~ / - no revision\(s\) after that date\./. The downside is we don't know how "stable" that message is. Will future versions of Perforce emit this message exactly, or is it possible they would reword?
Another option is to use the -s switch: my $p4cmd = "p4 -s files -e $files\#$previous_date,\#now";. That causes p4 to prepend the "severity" before every line of output. If a file is found, the line will start with info:, while the "no revision(s) after that date" will start with error:. That looks a bit more stable to me: exit if grep /^error:/, #filelist. Watch out for the last line; when you use the -s switch, you get an extra line with the exit code.
Yet another option would be to use P4Perl. In that case you'd get the results as structured data, which will obviate the parsing. That's arguably the most elegant, but you'd need the P4Perl module first.
I suggest using the -F flag to tame the output:
my $p4cmd = "p4 -F %depotFile% files -e $files\#$previous_date,\#now";
and then go ahead with the:
my #filelist = `$p4cmd`;
good_bye() unless #filelist; # Say goodbye and exit.
#filelist will be empty if there are no lines of output containing a %depotFile% field, and now your caller doesn't need to try to parse the depot path out of the standard p4 files output.
If you want to massage the p4 files output further, take a look at p4 -e files (args) so you can see what the different fields are that you can plug into -F.
Just do nothing if the array isn't populated.
my #filelist = `$p4cmd`;
good_bye() unless #filelist; # Say goodbye and exit.
chomp #filelist;
To suppress the message, just redirect stderr of the command to a bitbucket:
my $p4cmd = "p4 files -e $files\#$previous_date,\#now 2> /dev/null";
I found some command line with Perl that inserts headers into my files without going through the tedious process of inserting them one by one. Can someone walk me through the Perl aspect of this command line? I'm new to this and can't seem to find the right explanations for what I wrote.
cat header.txt | perl -0 -i -pe 'BEGIN{$h = <STDIN>}; print $h' 1*
-e
rather than provide a script in a xxxx.pl file, provide it on the command line
-p
makes it iterate over filename arguments somewhat like sed but also prints the contents of $_ at the end of the script.
the two above are combined in -pe
-i
indicate you want to edit the file in place and write the output to the same file. In practice, Perl renames the input file and reads from this renamed version while writing to a new file with the original name
-0
redefines the end of record character (\n by default) so that you can read the entire input file as a single line
1*
is the command line argument to your script, so I guess you are modifying any file with a name that starts with 1 (you could have used *.c, or whatever depending on the type of files you are trying to modify)
print $h
prints the variable $h that is the "main" of your script. if it was initialized with the content of the header file (the intent of this one-liner) then it will print the header file
BEGIN{ some code here }
this is stuff you execute before the script starts. this is where I'm stumped. this doesn't seem like valid perl code
so basically:
this will supposedly slurp the entire header file (because of -0) in the BEGIN block and store it in the variable $h
iterate over all the files specified by the wildcards at the end of the command line
for each file: print the header (print $h) then print hte file itself (because of -pe)
so it's equivalent to spelling the script out:
$h = gets content of the entire header file
while (<>){ #loop implied by -pe, iterates over all the 1* files
# the main contents of the "-e" script are inserted below as part of executing -pe
print h$; #print the header we saved
print $_; # implied by -pe, and since we are using -0, this prints the entire content in one shot
# end of the "-e" script. again it was a single print $h statement, the second print is implied by -pe
}
It's a bit hard to explain, take a look at the perlrun documentation for details (run man perlrun).
This is not 100% complete explanation because I don;t think the BEGIN block is right. I tried it on my ubuntu machine and it complained about its syntax too
Here's something similar, with an explanation. The program in the question doesn't run on my mac.
I needed to add the #nullable disable directive to the top of all my csharp files as part of migrating to nullable reference types.
perl -w -i -p -0777 -e 's/^/#nullable disable\n\n/' $(find . -iname '*.cs')
-w enable warnings
-i edit files in place
-p read each file block by block, printing each block after applying a perl expression. the default block size is one line
-0777 changes the default block size to the entire file
-e the perl expression to execute
The final argument uses shell command substitution to create a list of files. It passes that list of file paths to the perl command. The find command searches for files that end in .cs.
The perl program is a single substitution command. It matches the very beginning of the block and replaces (prepends, really) with "#nullable disable" and a couple new-lines.
In linux, to concatenate all files under a folder, you can do file=FOLDER/*; cat $file > ONEFILE, I also want to use this in my perl script so I coded like system("file=$folder/*");
system("cat \$file > $out");
But it won't work when I run the perl program, the $out was assigned a file name as my $out = "outfile";. The outfile always keeps at 0 bit. What's wrong here.
The first line sets the $file environment variable in a new shell process:
system "file=$folder/*";
The second line starts a new shell process with a new environment:
system "cat \$file > $out";
Since it's a new process, with a new environment, your previous $file variable is no longer set, so you are really running the following shell command:
cat > $out
Do this instead:
system "cat '$folder/'* > '$out';
Note - I also added quotes, which will help if your paths may contain spaces. However, it's still not safe against all forms of input, so don't pass any user input to that command without validating it first.
What's about exec in Perl?
perl -e 'exec "cat *.txt"'
I'm trying to write a perl script which takes the output of colorgcc (or any other script that prints colored text to terminal), adds/removes parts of the string, and then prints the result in the same color as the input string.
The following code will print "Hello World" in front of each line produced by the color_producing_script. The output will be all black, while the input is multicolored. How can I modified this script to conserve the original colors?
open(CMD, "color_producing_script |");
while(<CMD>) {
print 'Hello World' . $_;
}
I'm using bash terminal.
Edit
Per the excellent first comment, this is not a Perl issue per se. Just running color_producing_script | cat strips the color off. So the question can be rephrased to "How do you force color through the pipe?"
Edit 2
It looks like the latest version of gcc (1.3.2) includes the CGCC_FORCE_COLOR environment variable in the if clause, and if it's defined, colorgcc forces color:
export CGCC_FORCE_COLOR=true
Does color_producing_script change its behavior when it's used in a pipe? Try
color_producing_script | cat
at the command line. It may have an option to force color output even when it is.
The Perl script, colorgcc, is specifically checking to see if output is to a non-tty and skipping the colorization if that's the case.
# Get the terminal type.
$terminal = $ENV{"TERM"} || "dumb";
# If it's in the list of terminal types not to color, or if
# we're writing to something that's not a tty, don't do color.
if (! -t STDOUT || $nocolor{$terminal})
{
exec $compiler, #ARGV
or die("Couldn't exec");
}
Edit:
You could modify the script in one or more of the following ways:
comment out the test and make it always produce color output
add functionality to support reading an environment variable that sets whether to colorize
add functionality to support a color-always option in the ~/.colorgccrc configuration file
add functionality to support a color-always command line option that you strip before passing the rest of the options to the compiler
You could also use the expect script unbuffer to create a pseudo-tty like this:
unbuffer gcc file.c | cat
(where cat is a standin recipient).
All of this is based on using colorgcc from the command line. There should be analogs for doing similar things within a Perl script.
Many programs that generate colored output detect if they're writing to a TTY, and switch off colors if they aren't. This is because color codes are annoying when you only want to capture the text, so they try to "do the right thing" automatically.
The simplest way to capture color output from a program like that is to tell it to write color even though it's not connected to a TTY. You'll have to read the program's documentation to find out if it has that option.
The other option is to use a Pty instead of a pipe. In Perl, you can do this with IO::Pty::HalfDuplex or IO::Pty::Easy, both of which are higher-level wrappers around the low-level module IO::Pty.
The source code of ColorGCC is quite clear about this topic!
#! /usr/bin/perl -w
# ...
# from: colorgcc-4.1.2/colorgcc-4.1.2
# downloaded from: http://www.console-colors.de/index.php?n=ConsColors.Downloads
#
# Note:
#
# colorgcc will only emit color codes if:
#
# (1) Its STDOUT is a tty and
# (2) the value of $TERM is not listed in the "nocolor" option.
#
# If colorgcc colorizes the output, the compiler's STDERR will be
# combined with STDOUT. Otherwise, colorgcc just passes the output from
# the compiler through without modification.
# .....
# If it's in the list of terminal types not to color, or if
# we're writing to something that's not a tty, don't do color.
if (! -t STDOUT || $nocolor{$terminal})
{
exec $compiler, #ARGV
or die("Couldn't exec");
}
In addition to use a Pty instead of a pipe in Perl (as already pointed out by cjm) you can trick an application into thinking its stdin is interactive, not a pipe on the command line as well.
For example:
# Linux
script -c "[executable string]" /dev/null
# FreeBSD, Mac OS X
script -q /dev/null "[executable string]"
For further solutions see: bash: force exec'd process to have unbuffered stdout