Perl script to rename files with spaces in name, pushd/popd equivalent? - perl

My Linux system mounts some Samba shares, and some files are deposited by Windows users. The names of these files sometimes contain spaces and other undesirable characters. Changing these characters to hyphens - seems like a reasonable solution. Nothing else needs to be changed to handle these cleaned file names.
A couple of questions,
What other characters besides spaces, parenthesis should be translated?
What other file attributes (besides file type (file/dir) and permissions) should be checked?
Does Perl offer a pushd/popd equivalent, or is chdir a reasonable solution to traversing the directory tree?
This is my Perl program
#!/bin/env perl
use strict;
use warnings;
use File::Copy;
#rename files, map characters (not allowed) to allowed characters
#map [\s\(\)] to "-"
my $verbose = 2;
my $pat = "[\\s\\(\\)]";
sub clean {
my ($name) = #_;
my $name2 = $name;
$name2 =~ s/$pat/\-/g;
#skip when unchanged, collision
return $name if (($name eq $name2) || -e $name2); #skip collisions
print "r: $name\n" if ($verbose > 2);
rename($name, $name2);
$name2;
}
sub pDir {
my ($obj) = #_;
return if (!-d $obj);
return if (!opendir(DIR, $obj));
print "p: $obj/\n" if ($verbose > 2);
chdir($obj);
foreach my $dir (readdir DIR) {
next if ($dir =~ /^\.\.?$/); #skip ./, ../
pDir(clean($dir));
}
close(DIR);
chdir("..");
}
sub main {
foreach my $argv (#ARGV) {
print "$argv/\n" if ($verbose > 3);
$argv = clean($argv);
if (-d $argv) { pDir($argv); }
}
}
&main();
These posts are related, but don't really address my questions,
Use quotes: How to handle filenames with spaces? (using other scripts, prefer removing need for quotes)
File::Find perl script to recursively list all filename in directory (yes, but I have other reasons)
URL escaping: Modifying a Perl script which has an error handling spaces in files (not urls)
Quotemeta: How can I safely pass a filename with spaces to an external command in Perl? (not urls)

Here's a different way to think about the problem:
Perl has a built-in rename function. You should use it.
Create a data structure mapping old names to new names. Having this data will allow various sanity checks: for example, you don't want cleaned names stomping over existing files.
Since you aren't processing the directories recursively, you can use glob to good advantage. No need to go through the hassles of opening directories, reading them, filtering out dot-dirs, etc.
Don't invoke subroutines with a leading ampersand (search this issue for more details).
Many Unix-like systems include a Perl-based rename command for quick-and-dirty renaming jobs. It's good to know about even if you don't use it for your current project.
Here's a rough outline:
use strict;
use warnings;
sub main {
# Map the input arguments to oldname-newname pairs.
my #renamings =
map { [$_, cleaned($_)] }
map { -f $_ ? $_ : glob("$_/*") }
#_;
# Sanity checks first.
# - New names should be unique.
# - New should not already exist.
# - ...
# Then rename.
for my $rnm (#renamings){
my ($old, $new) = #$rnm;
rename($old, $new) unless $new eq $old;
}
}
sub cleaned {
# Allowed characters: word characters, hyphens, periods, slashes.
# Adjust as needed.
my $f = shift;
$f =~ s/[^\w\-\.\/]/-/g;
return $f;
}
main(#ARGV);

Don't blame Windows for your problems. Linux is much more lax, and the only character it prohibits from its file names is NUL.
It isn't clear exactly what you are asking. Did you publish your code for a critique, or are you having problems with it?
As for the specific questions you asked,
What other characters besides spaces, parenthesis should be translated?
Windows allows any character in its filenames except for control characters from 0x00 to 0x1F and any of < > \ / * ? |
DEL at 0x7F is fine.
Within the ASCII set, that leaves ! # $ % & ' ( ) + , - . : ; = # [ ] ^ _ ` { } ~
The set of characters you need to translate depends on your reason for doing this. You may want to start by excluding non-ASCII characters, so your code should read something like
$name2 =~ tr/\x21-\x7E/-/c
which will change all non-ASCII characters, spaces and DEL to hyphens. Then you need to go ahead and fix all the ASCII characters that you consider undersirable.
What other file attributes (besides file type (file/dir) and permissions) should be checked?
The answer to this has to be according to your purpose. If you are referring only to whether renaming a file or directory as required is possible, then I suggest that you just let rename itself tell you whether it succeeded. It will return a false value if the operation failed, and the reason will be in $!.
Does Perl offer a pushd/popd equivalent, or is chdir a reasonable solution to traversing the directory tree?
If you want to work with that idiom, then you should take a look at File::pushd, which allows you to temporarily chdir to a new location. A popd is done implicitly at the end of the enclosing block.
I hope this helps. If you have any other specific questions then please make them known by editing your original post.

Related

Get the path for a similarly named file in perl, where only the extension differs?

I'm trying to write an Automator service, so I can chuck this into a right-click menu in the gui.
I have a filepath to a txt file, and there is a similarly named file that varies only in the file extension. This can be a pdf or a jpg, or potentially any other extension, no way to know beforehand. How can I get the filepath to this other file (there will only be one such)?
$other_name =~ s/txt$/!(txt)/;
$other_name =~ s/ /?/g;
my #test = glob "$other_name";
In Bash, I'd just turn on the extglob option, and change the "txt" at the end to "!(txt)" and the do glob expansion. But I'm not even sure if that's available in perl. And since the filepaths always have spaces (it's in one of the near-root directory names), that further complicates things. I've read through the glob() documentation at http://perldoc.perl.org/functions/glob.html and tried every variation of quoting (the above example code shows my attempt after having given up, where I just remove all the spaces entirely).
It seems like I'm able to put modules inside the script, so this doesn't have to be bare perl (just ran a test).
Is there an elegant or at least simple way to accomplish this?
You can extract everything in the filename up to extension, then run a glob with that and filter out the unneeded .txt. This is one of those cases where you need to protect the pattern in the glob with a double set of quotes, for spaces.
use warnings;
use strict;
use feature qw(say);
my $file = "dir with space/file with spaces.txt";
# Pull the full name without extension
my ($basefname) = $file =~ m/(.*)\.txt$/;
# Get all files with that name and filter out unneeded (txt)
my #other_exts = grep { not /\.txt$/ } glob(qq{"$basefname.*"});
say for #other_exts;
With a toy structure like this
dir space/
file with spaces.pdf
file with spaces.txt
The output is
dir space/file with spaces.pdf
This recent post has more on related globs.
Perl doesn't allow the not substring construct in glob. You have to find all files with the same name and any extension, and remove the one ending with .txt
This program shows the idea. It splits the original file name into a stem part and a suffix part, and uses the stem to form a glob pattern. The grep removes any result that ends with the original suffix
It picks only the first matching file name if there is more than one candidate. $other_name will be set to undef if no matching file was found
The original file name is expected as a parameter on the command line
The result is printed to STDOUT; I don't know what you need for your right-click menu
The line use File::Glob ':bsd_glob' is necessary if you are working with file paths that contain spaces, as it seems you are
use strict;
use warnings 'all';
use File::Glob ':bsd_glob';
my ($stem, $suffix) = shift =~ /(.*)(\..*)/;
my ($other_name) = grep ! /$suffix$/i, glob "$stem.*";
$other_name =~ tr/ /?/;
print $other_name, "\n";
This is an example, based on File::Basename core module
use File::Basename;
my $fullname = "/path/to/my/filename.txt";
my ($name, $path, $suffix) = fileparse($fullname, qw/.txt/);
my $new_filename = $path . $name . ".pdf";
# $name --> filename
# $path --> /path/to/my/
# $suffix --> .txt
# $new_filename --> /path/to/my/filename.pdf

How do I detect a case-insensitive file system in Perl?

I tried using File::Spec->case_tolerant, but it returns false on HFS+, which is wrong. I suspect it's because File::Spec::Unix always returns false. My current workaround is this function:
my $IS_CASE_INSENSITIVE;
sub _is_case_insensitive {
unless (defined $IS_CASE_INSENSITIVE) {
$IS_CASE_INSENSITIVE = 0;
my ($uc) = glob uc __FILE__;
if ($uc) {
my ($lc) = glob lc __FILE__;
$IS_CASE_INSENSITIVE = 1 if $lc;
}
}
return $IS_CASE_INSENSITIVE;
}
But that's a hack since: 1) on a case-sensitive file system both of those files might exists; and 2) different volumes can have different file systems.
In truth, every directory considered must be checked on its own. This is because, on Unix-like systems, any directory can be a different file system than some other directory. Furthermore, use of glob is not very reliable; from perlport:
Don't count on filename globbing. Use opendir, readdir, and closedir instead.
But I think that #borodin is onto something with the use of -e. So here's a function that uses -e to determine whether the specified directory is on a case-insensitive file system:
my %IS_CASE_INSENSITIVE;
sub is_case_insensitive {
my $dir = shift;
unless (defined $IS_CASE_INSENSITIVE{$dir}) {
$IS_CASE_INSENSITIVE{$dir} = -e uc $dir && -e lc $dir;
}
return $IS_CASE_INSENSITIVE{$dir};
}
You could probably add some heuristics for Windows to just cache the value for the drive letter, since that defines a mount point. And of course, it will fail on case-sensisitve file systems if both uppercase and lowercase variations of the directory exist. But otherwise, unless there is some other way to tell more globally which directories match to which mount points, you have to check for any directory.
I suggest you make use of the core File::Temp module to create a new unique file that has lower-case characters in its name. The file is set to be deleted when the object is destroyed, which is when the subroutine exits if not before.
If the file doesn't exist when access by the upper-cased file name then the filing system is case-sensitive.
If the upper-cased name does exist then we have to check that we haven't happened upon a file whose upper-case version was already there, so we delete the file. If the upper-case entry has now gone then the filing system is case-insensitive.
If the upper-cased name is still there then it is a file that existed before we created the temporary file. We just loop around and create a new temporary file with a different name, although the chances of this happening are absolutely tiny. If you prefer you can minimize this possibility even further by using an outlandish value for SUFFIX. Just be careful that the characters you use are valid on any fileing system that you wish to test.
I've tested this on both Windows 7 and Ubuntu.
use strict;
use warnings;
use 5.010;
use autodie;
use File::Temp ();
printf "File system %s case_insensitive\n", case_insensitive() ? "is" : "isn't";
sub case_insensitive {
while () {
my $tmp = File::Temp->new(
TEMPLATE => 'tempXXXXXX',
SUFFIX => '.tmp',
UNLINK => 1,
);
my $uc_filename = uc $tmp->filename;
return 0 if not -e $uc_filename;
$tmp = undef;
return 1 if not -e $uc_filename;
}
}

PERL: String Replacement on file

I am working on a script to do a string replacement in a file and I will read the variables and values and files from a configuration file and do string replacement.
Here is my logic to do a string replacement.
sub expansion($$$){
my $f = shift(#_) ; # file Name
my $vname = shift(#_) ; # variable name for pattern match
my $value = shift(#_) ; # value to replace
my $n = "$f".".new";
open ( O, "<$f") or print( "Can't open $f file: $!");
open ( N ,">$n" ) or print( "Can't open $n file: $!");
while (<O>)
{
$_ =~ s/$vname/$value/g; #check for pattern
print N "$_" ;
}
close (O);
close (N);
}
In my logic am reading line by line in from input file ($f) for the pattern and writing to a new file ($n) .
Instead of write to a new file is there any way to do a string replacement the original file when I try to do the same it has only empty file with no contents.
Do not. Never, ever1. Don't you dare, Don't even think of, do not use subroutine prototyping. It is horribly broken (that is, it doesn't do what you think it does) and is dangerous.
Now, we got that out of the way:
Yes, you can do what you want. You can open a file as both read and writable by using the mode <+. So far, so good.
However, due to buffering, you cannot use the standard read and write methods to read and write to the file. Instead, you need to use sysread and syswrite.
Then, what you need to do is read the line, use sysseek to go back to the start of where you read, and then write to that spot.
Not only is it very complex to do, but it is full of peril. Let's take a simple example. I have a document, and I want to replace my curly quotes with straight quotes.
$line =~ s/“|”/"/g;
That should work. I'm replacing one character with another. What could go wrong?
If this is a UTF-8 file (what Macs and Linux systems use by default), those curly quotes are two-byte characters and that straight quote is a single byte character. I would be writing back a line that was shorter than the line I read in. My buffer is going to be off.
Back in the days when computer memory and storage were measured in kilobytes, and you serial devices like reel-to-reel tapes, this type of operation was quite common. However, in this age where storage is vast, it's simply not worth the complexity and error prone process that this entails. Stick with reading from one file, and writing to another. Then use unlink and rename to delete the original and to rename the copy to the original's name.
A few more pointers:
Don't print if the file can't be opened. Use die. Otherwise, your program will simply continue on blithely unaware that it is not working. Even better, use the pragma use autodie;, and you won't have to worry about testing whether or not a read/write failed.
Use scalars for file handles.
That is instead of
open OUT, ">my_file.txt";
use
open my $out_fh, ">my_file.txt";
And, it is highly recommended to use the three parameter open:
Use
open my $out_fh, ">", "my_file.txt";
If you aren't, always add use strict; and use warnings;.
In fact, your Perl syntax is a bit ancient. You need to get a book on Modern Perl. Perl originally was written as a hack language to replace shell and awk programming. However, Perl has morphed into a full fledge language that can handle complex data types, object orientation, and large projects. Learning the modern syntax of Perl will help you find errors, and become a better developer.
1. Like all rules, this can be broken, but only if you have a clear and careful understanding what is going on. It's like those shows that say "Don't do this at home. We're professionals."
sub inplace_expansion($$$){
my $f = shift(#_) ; # file Name
my $vname = shift(#_) ; # variable name for pattern match
my $value = shift(#_) ; # value to replace
local #ARGV = ( $f );
local $^I = '';
while (<>)
{
s/\Q$vname/$value/g; #check for pattern
print;
}
}
or, my preference would run closer to this (basically equivalent, changes mostly in formatting, variable names, etc.):
use English;
sub inplace_expansion {
my ( $filename, $pattern, $replacement ) = #_;
local #ARGV = ( $filename ),
$INPLACE_EDIT = '';
while ( <> ) {
s/\Q$pattern/$replacement/g;
print;
}
}
The trick with local basically simulates a command-line script (as one would run with perl -e); for more details, see perldoc perlrun. For more on $^I (aka $INPLACE_EDIT), see perldoc perlvar.
(For the business with \Q (in the s// expression), see perldoc -f quotemeta. This is unrelated to your question, but good to know. Also be aware that passing regex patterns around in variables—as opposed to, e.g., using literal regexes exclusively— can be vulnerable to injection attacks; Perl's built-in taint mode is useful here.)
EDIT: David W. is right about prototypes.

Can NOT List directory including space using Perl in Windows Platform

In order to list pathes in Windows,I wrote below Perl function(executed under StrawBerry runtime environment).
sub listpath
{
my $path = shift;
my #list = glob "$path/*";
#my #list = <$path/*>;
my #pathes = grep { -d and $_ ne "." and $_ ne ".." } #list;
}
But it can't parse directory including space correctly, for example:
When I issued following code:
listpath("e:/test/test1/test11/test111/test1111/test11111 - Copy");
The function returned an array including two elements:
1: e:/test/test1/test11/test111/test1111/test11111
2: -
I am wondering if glob could parse above space directories. Thanks a lot.
Try bsd_glob instead:
use File::Glob ':glob';
my #list = bsd_glob "$path/*";
Even if the topic has been answered long time ago, I recently encounter the same problem, and a quick search gives me another solution, from perlmonks (last reply):
my $path = shift;
$path =~ s/ /\\ /g;
my #list = glob "$path/*";
But prefer bsd_glob, it supports also a couple of other neat features, such as [] for character class.
The question is about Windows platform, where Bentoy13's solution does not work because the backslash would be mistaken for a path separator.
Here's an option if for whatever reason you don't want to go with bsd_glob: wrap the offensive part of the path in double quotes. This can be one directory name (path\\"to my"\\file.txt) or several directory names ("path\\to my"\\file.txt). Slash instead of backslash usually works, too. Of course, they don't have to include a space, so this here always works:
my #list = glob "\"$path\"/*";
remember, it's a Windows solution. Whether it works under Linux depends on context.

How to handle filenames with spaces?

I use Perl on windows(Active Perl). I have a perl program to glob the files in current folder, and concatenate them all using dos copy command called from within using system()...
When i execute, this gives a dos error saying "The system cannot find the file specified." It's related to the spaces in the filenames I have.
This is the perl code :-
#files = glob "*.mp3";
$outfile = 'final.mp3';
$firsttime = 1;
foreach (#files)
{
if($firsttime == 1)
{
#args = ('copy' ,"/b ","$_","+","$outfile", "$outfile");
system (#args);
#system("copy /b '$_'+$outfile $outfile");
$firsttime = 0;
}
else
{
#args = ('copy' ,"/b ","$outfile","+","$_", "$outfile");
system (#args);
#system("copy /b $outfile+'$_' $outfile");
}
}
glob returns a array of filenames in my current folder, Those file names have spaces in between them, so the array elements have spaces in between. When i use the system(...) to execute my copy command on those array elements using "$_" as shown above, it gives error as above.
I tried couple of ways in which I could call the system(...) but without any success.
I would like to know,
1] How can i get this working on files which have spaces in between them using the code above. How to 'escape' the white space in file names.
2] Any alternative solution in Perl to achieve the same thing. (Simple ones welcome..)
Stop using system() to make a call that can be done with a portable library. Perl has a the File::Copy module, use that instead and you don't have to worry about things like this plus you get much better OS portability.
Your code doesn't add any quotes around the filenames.
Try
"\"$_\""
and
"\"$outfile\""
system is rarely the right answer, use File::Copy;
To concatenate all files:
use File::Copy;
my #in = glob "*.mp3";
my $out = "final.mp3";
open my $outh, ">", $out;
for my $file (#in) {
next if $file eq $out;
copy($file, $outh);
}
close $outh;
Issues may arise when you're trying to access the variable $_ inside an inner block. The safest way, change:
foreach (#files)
to:
foreach $file (#files)
Then do the necessary changes on #args, and escape doublequotes to include them in the string..
#args = ('copy' ,"/b ","\"$file\"","+","$outfile", "$outfile");
...
#args = ('copy' ,"/b ","$outfile","+","\"$file\"", "$outfile");
In windows you can normally put double quotes around the filenames (and/or paths) allowing special chars i.e "long file names".
C:\"my long path\this is a file.mp3"
Edit:
Does this not work?
system("copy /b \"$_\"+$outfile $outfile");
(NOTE THE DOUBLE quotes within the string not single quotes)
$filename =~ s/\ /\ /;
what ever the filename is just use slash to refrence spaces
The built in "rename" command also moves files:
rename $source, $destination; # ...and move
I use this on windows all the time.