I am attempting to write a small PowerShell script to clean up files names from some log dumps, but I seem to be stuck... I have logs dumped from various sources, and the file names seem to be getting garbled up.
I am looking to for name the names of files like so... " Source - Service.log "
Get-ChildItem *.* -Path ~/Desktop/New | ForEach-Object {
while ([string]($_.Name) -notmatch "^[a-z].*" -or [string]($_.Name) -notmatch "^[A-Z].*") {
Rename-Item -NewName { [string]($_.Name).Substring(1) }
}
Write-Host $_.Name
}
The output seems to be erroring out.
Rename-Item : Cannot evaluate parameter 'NewName' because its argument is
specified as a script block and there is no input. A script block cannot be
evaluated without input.
At line:8 char:30
+ Rename-Item -NewName { $File.Substring(1) }
+ ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : MetadataError: (:) [Rename-Item], ParameterBindingException
+ FullyQualifiedErrorId : ScriptBlockArgumentNoInput,Microsoft.PowerShell.Commands.RenameItemCommand
The Idea is to check the filename to to see if it is a character, and if not remove it, to remove ". - / and whitespace"
The orginal files I am running against are like this:
1. source - data (1).log
100. - source - Data.log
(1) Source - data.log
source - data.log
<space><space> source - data.log
and the result I am looking for from the above is: I am not concerned about the duplicates file names as source and data change day to day and the folder is cleared regularly...
source - data (1).log
source - Data.log
Source - data.log
source - data.log
source - data.log
Can someone tell me how to get past this error?
If your goal is to delete leading non-alpha characters, you can simplify what you're doing:
$files = Get-ChildItem -Path ~\Desktop\New -File
foreach ($file in $files)
{
if ($file.BaseName -notmatch '\S+\s-')
{
$newName = $file.Name -replace '^.+?(?=[a-z])'
$newName = Join-Path $file.DirectoryName $newName
if (Test-Path -Path $newName)
{
Remove-Item -Path $newName
}
$file | Rename-Item -NewName $newName
Write-Verbose $newName
}
}
This will iterate your list and look for your pattern, renaming where necessary. Assumption: the source doesn't have spaces.
This might be helpful: Remove-NonAlphanumericCharFromString.
Knowing how to remove non-alphanumeric, take the base name of the file (name without Path and extension).
Replace unwanted Chars with empty string.
$pattern = '[^a-zA-Z]'
Set-Location <YourDir>
Get-Childitem | Foreach-Object {
Rename-Item -Path ".\$($_.Name)" -NewName "$($_.BaseName -replace $pattern,'')$($_.extension)"
}
Note that above will fail upon a need to overwrite existing file.
Related
I modified PowerShell script from PowerShell - Batch change files encoding To UTF-8.
# Modified version of https://stackoverflow.com/q/18684793
[Threading.Thread]::CurrentThread.CurrentUICulture = 'en-US'
$Encoding = New-Object System.Text.UTF8Encoding($True) # If UTF8Encoding($False), It will be UTF-8 without BOM
$source = "C:\Users\AKULA\Desktop\SRC" # source directory
$destination = "C:\Users\AKULA\Desktop\DST" # destination directory
if (!(Test-Path $destination)) {
New-Item -Path $destination -ItemType Directory | Out-Null
}
# Delete all previously generated file
Get-ChildItem -Path $destination -Include * -File -Recurse | ForEach-Object {$_.Delete()}
# Recursively convert all files into UTF-8
foreach ($i in Get-ChildItem $source -Force -Recurse -Exclude "desktop.ini") {
if ($i.PSIsContainer) {
continue
}
$name = $i.Fullname.Replace($source, $destination)
$content = Get-Content $i.Fullname
if ($null -ne $content) {
[System.IO.File]::WriteAllLines($name, $content, $Encoding)
} else {
Write-Host "No content from: $i"
}
}
But after using it, I've found that PS cannot handle [ or ] well.
I made some test files that has diversity in name/content.
Get-Content : An object at the specified path C:\Users\AKULA\Desktop\SRC\FILENAME[[[[[[]]]]]]]].txt does not exist, or
has been filtered by the -Include or -Exclude parameter.
At C:\Users\AKULA\Desktop\Convert_to_UTF-8.ps1:24 char:16
+ $content = Get-Content $i.Fullname
+ ~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (System.String[]:String[]) [Get-Content], Exception
+ FullyQualifiedErrorId : ItemNotFound,Microsoft.PowerShell.Commands.GetContentCommand
Since I cannot embed images in question, here is link of IMGUR album.
Full image list: https://imgur.com/a/aN1RG2L
These are what I've tested:
Test files have different names. Their name contains space, ',
[]. Also made up different language(Japanese, Korean).
These files have same content, encoded with UCS-2 BE BOM(UTF-16 BE) so
that I can check if it has re-encoded to UTF-8.
How can I make my script handle [ or ] in file name well?
tl;dr
Indeed, use of the -LiteralPath parameter is the best solution (in PowerShell (Core) v6+, you can shorten to -lp):
$content = Get-Content -LiteralPath $i.Fullname
-LiteralPath ensures that $i.Fullname is taken verbatim (literally); that is, [ and ] in the path are interpreted as themselves rather than having special meaning, as they would have as a -Path argument, due to being interpreted as a wildcard expression - note that -Path is positionally implied if you only pass a value (a string) as the first argument, as you did (Get-Content $i.FullName)
Note: This answer analogously applies to all cmdlets that have both -Path and -LiteralPath parameters, such as Set-Content, Out-File, and Set-Location.
As for what you tried:
$content = Get-Content $i.Fullname
is effectively the same as:
$content = Get-Content -Path $i.Fullname
That is, the (first) positional argument passed to Get-Content is implicitly bound to the
-Path parameter.
The -Path parameter accepts wildcard expressions to allow matching paths by patterns; in addition to support for * (any run of characters) and ? (exactly 1 character), [...] inside a wildcard pattern denotes a character set or range (e.g., [12] or [0-9]).
Therefore an actual path that contains [...], e.g., foo[10].txt, is not recognized as such, because the [10] is interpreted as a character set matching a single character that is either 1 or 0; that is foo[10].txt would match foo0.txt and foo1.txt, but not a file literally named foo[10].txt.
When (implicitly) using -Path, it is possible to escape [ and ] instances that should be interpreted verbatim, namely via the backtick (`), but note that this can get tricky to get right when quoting and/or variable references are involved.
If you know a path to be a literal path, it is best to form a habit of using -LiteralPath (which in PowerShell Core you can shorten to -lp).
However, if your path contains literal [ and ] and you also need wildcard matching, you must use `-escaping - see this answer.
There are at least two situations where the solution's good advice doesn't hold, unfortunately.
Selective error handling
Get-Content -LiteralPath "nobox[]" gives an error message and exception type as if wildcards are involved:
Get-Content : An object at the specified path box[] does not exist, or has been filtered by the -Include or -Exclude parameter.
At line:1 char:1
+ Get-Content -Path "nobox[]"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (System.String[]:String[]) [Get-Content], Exception
+ FullyQualifiedErrorId : ItemNotFound,Microsoft.PowerShell.Commands.GetContentCommand
whereas without the brackets, we get:
Get-Content : Cannot find path 'nobox' because it does not exist.
At line:1 char:1
+ Get-Content -LiteralPath "nobox"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (nobox:String) [Get-Content], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand
Therefore, to silently deal with an optional file, something like:
try {
$lines = Get-Content -LiteralPath $path -ErrorAction Stop
}
catch [System.Management.Automation.ItemNotFoundException] {
$lines = #()
}
chokes on paths with brackets.
Creating a hard or symbolic link
A minor and a major caveat:
The Path parameter, the name of the new item, "works like the LiteralPath parameter of other cmdlets", says the documentation of New-Item clearly, and that seems true and makes sense. Though I wish we could clarify that by writing -LiteralPath.
The Value parameter, the target of the link (also known as Target secretly in v5 and openly later), does not accept wildcard characters according to the same documentation, but that's a lie. The command:
New-Item -ItemType "HardLink" -Path "whatever" -Target "*"
makes Powershell squeal "Cannot set the location because path '*' resolved to multiple containers.".
So you always need the escapes for the target. If you have a file named "f[]", then this will display an error:
New-Item -ItemType "HardLink" -Path "whatever" -Target "f[]"
and this will create a link:
New-Item -ItemType "HardLink" -Path "f[2]" -Target ([WildcardPattern]::Escape("f[]"))
Same for ItemType "SymbolicLink".
I have some long filenames that potensially can cause issues with Windows path max chars, and I would like to rename them preserving part of it - and also adding a RNG 4 letter/digit combination.
Filename example: 478432_1400_79834_SomeKindofText_UserInputSoItCanBeReallyLongCombinedWithANetworkPath.jpg
Wanted rename outcome:
478432_1400_79834_SomeKindofText_abc1.jpg
Where 'abc1' represents the 4 letter/digit combination of a checksum
This is the code I have so far:
$search_folder = "C:\PS\Test\"
Get-ChildItem $search_folder -File | ForEach-Object {
$checksum = Get-FileHash -Path $_
$checksum = $checksum.substring(0,3)
Rename-Item -NewName { $search_folder+$_.BaseName.Split('_')[0..3] + $checksum + $_.Extension }
}
My first problem is that Get-FileHash does not support substring method, generating a error message:
Method invocation failed because [Microsoft.Powershell.Utility.FileHash] does not contain a method named 'substring'.
My second problem is that it tries to do a Resolve-Path in my current PS shell directory instead of $search_folder
My third problem is that the underscores in the filename is not preserved, so a -WhatIf tag on the Rename-Item method yields a result like "478432 1400 79834 SomeKindofText"
Tips or suggestions would be most welcomed!
My first problem is that Get-FileHash does not support substring method, generating a error message:
Method invocation failed because [Microsoft.Powershell.Utility.FileHash] does not contain a method named 'substring'.
$checksum does not store the hash string, it stores an object that has a property named Hash, which in turn stores the string you want, so change this line:
$checksum = $checksum.substring(0,3)
To:
$checksum = $checksum.Hash.Substring(0,3)
My second problem is that it tries to do a Resolve-Path in my current PS shell directory instead of $search_folder
Two general solutions to this problem:
Pass the absolute path to the file explicitly:
Rename-Item -LiteralPath $_.FullName -NewName { ... }
Or pipe the output from Get-ChildItem directly to Rename-Item and let PowerShell bind the path correctly:
$_ |Rename-Item -NewName { ... }
My third problem is that the underscores in the filename is not preserved, so a -WhatIf tag on the Rename-Item method yields a result like "478432 1400 79834 SomeKindofText"
Splitting a string on '_' will also remove the underscores - to reverse this, use '_' as a delimiter in a -join operation:
$firstFourPartsOfBaseName = $_.BaseName.Split('_')[0..3] -join '_'
Putting this all together, we get:
$search_folder = "C:\PS\Test\"
Get-ChildItem $search_folder -File | ForEach-Object {
$checksum = Get-FileHash -Path $_
$checksum = $checksum.hash.substring(0,3)
$_ |Rename-Item -NewName {
# calculate new base name ("478432_1400_79834_SomeKindofText_abc1")
$newBasename = #($_.BaseName.Split('_')[0..3]; $checksum) -join ''
# add extension and output
$newBasename,$_.Extension -join '.'
}
}
Please see your script adjusted below:
$search_folder = "C:\PS\Test\"
Get-ChildItem $search_folder -File | ForEach-Object {
$checksum = Get-FileHash -Path $_
$checksum = $checksum.hash.substring(0,3)
Rename-Item -Path $_ -NewName ($search_folder + $_.BaseName.Split('_')[0..3] + $checksum + $_.Extension)
}
You were trying to get a sub string of an object, using the property hash on $Checksum will allow you to create a substring.
I have also added -path to the rename-item request and changed the parenthesis on the string construction (it could be either of these that were causing you an issue, unfortunately I haven't tested this.)
I modified PowerShell script from PowerShell - Batch change files encoding To UTF-8.
# Modified version of https://stackoverflow.com/q/18684793
[Threading.Thread]::CurrentThread.CurrentUICulture = 'en-US'
$Encoding = New-Object System.Text.UTF8Encoding($True) # If UTF8Encoding($False), It will be UTF-8 without BOM
$source = "C:\Users\AKULA\Desktop\SRC" # source directory
$destination = "C:\Users\AKULA\Desktop\DST" # destination directory
if (!(Test-Path $destination)) {
New-Item -Path $destination -ItemType Directory | Out-Null
}
# Delete all previously generated file
Get-ChildItem -Path $destination -Include * -File -Recurse | ForEach-Object {$_.Delete()}
# Recursively convert all files into UTF-8
foreach ($i in Get-ChildItem $source -Force -Recurse -Exclude "desktop.ini") {
if ($i.PSIsContainer) {
continue
}
$name = $i.Fullname.Replace($source, $destination)
$content = Get-Content $i.Fullname
if ($null -ne $content) {
[System.IO.File]::WriteAllLines($name, $content, $Encoding)
} else {
Write-Host "No content from: $i"
}
}
But after using it, I've found that PS cannot handle [ or ] well.
I made some test files that has diversity in name/content.
Get-Content : An object at the specified path C:\Users\AKULA\Desktop\SRC\FILENAME[[[[[[]]]]]]]].txt does not exist, or
has been filtered by the -Include or -Exclude parameter.
At C:\Users\AKULA\Desktop\Convert_to_UTF-8.ps1:24 char:16
+ $content = Get-Content $i.Fullname
+ ~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (System.String[]:String[]) [Get-Content], Exception
+ FullyQualifiedErrorId : ItemNotFound,Microsoft.PowerShell.Commands.GetContentCommand
Since I cannot embed images in question, here is link of IMGUR album.
Full image list: https://imgur.com/a/aN1RG2L
These are what I've tested:
Test files have different names. Their name contains space, ',
[]. Also made up different language(Japanese, Korean).
These files have same content, encoded with UCS-2 BE BOM(UTF-16 BE) so
that I can check if it has re-encoded to UTF-8.
How can I make my script handle [ or ] in file name well?
tl;dr
Indeed, use of the -LiteralPath parameter is the best solution (in PowerShell (Core) v6+, you can shorten to -lp):
$content = Get-Content -LiteralPath $i.Fullname
-LiteralPath ensures that $i.Fullname is taken verbatim (literally); that is, [ and ] in the path are interpreted as themselves rather than having special meaning, as they would have as a -Path argument, due to being interpreted as a wildcard expression - note that -Path is positionally implied if you only pass a value (a string) as the first argument, as you did (Get-Content $i.FullName)
Note: This answer analogously applies to all cmdlets that have both -Path and -LiteralPath parameters, such as Set-Content, Out-File, and Set-Location.
As for what you tried:
$content = Get-Content $i.Fullname
is effectively the same as:
$content = Get-Content -Path $i.Fullname
That is, the (first) positional argument passed to Get-Content is implicitly bound to the
-Path parameter.
The -Path parameter accepts wildcard expressions to allow matching paths by patterns; in addition to support for * (any run of characters) and ? (exactly 1 character), [...] inside a wildcard pattern denotes a character set or range (e.g., [12] or [0-9]).
Therefore an actual path that contains [...], e.g., foo[10].txt, is not recognized as such, because the [10] is interpreted as a character set matching a single character that is either 1 or 0; that is foo[10].txt would match foo0.txt and foo1.txt, but not a file literally named foo[10].txt.
When (implicitly) using -Path, it is possible to escape [ and ] instances that should be interpreted verbatim, namely via the backtick (`), but note that this can get tricky to get right when quoting and/or variable references are involved.
If you know a path to be a literal path, it is best to form a habit of using -LiteralPath (which in PowerShell Core you can shorten to -lp).
However, if your path contains literal [ and ] and you also need wildcard matching, you must use `-escaping - see this answer.
There are at least two situations where the solution's good advice doesn't hold, unfortunately.
Selective error handling
Get-Content -LiteralPath "nobox[]" gives an error message and exception type as if wildcards are involved:
Get-Content : An object at the specified path box[] does not exist, or has been filtered by the -Include or -Exclude parameter.
At line:1 char:1
+ Get-Content -Path "nobox[]"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (System.String[]:String[]) [Get-Content], Exception
+ FullyQualifiedErrorId : ItemNotFound,Microsoft.PowerShell.Commands.GetContentCommand
whereas without the brackets, we get:
Get-Content : Cannot find path 'nobox' because it does not exist.
At line:1 char:1
+ Get-Content -LiteralPath "nobox"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (nobox:String) [Get-Content], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand
Therefore, to silently deal with an optional file, something like:
try {
$lines = Get-Content -LiteralPath $path -ErrorAction Stop
}
catch [System.Management.Automation.ItemNotFoundException] {
$lines = #()
}
chokes on paths with brackets.
Creating a hard or symbolic link
A minor and a major caveat:
The Path parameter, the name of the new item, "works like the LiteralPath parameter of other cmdlets", says the documentation of New-Item clearly, and that seems true and makes sense. Though I wish we could clarify that by writing -LiteralPath.
The Value parameter, the target of the link (also known as Target secretly in v5 and openly later), does not accept wildcard characters according to the same documentation, but that's a lie. The command:
New-Item -ItemType "HardLink" -Path "whatever" -Target "*"
makes Powershell squeal "Cannot set the location because path '*' resolved to multiple containers.".
So you always need the escapes for the target. If you have a file named "f[]", then this will display an error:
New-Item -ItemType "HardLink" -Path "whatever" -Target "f[]"
and this will create a link:
New-Item -ItemType "HardLink" -Path "f[2]" -Target ([WildcardPattern]::Escape("f[]"))
Same for ItemType "SymbolicLink".
I'm a Powershell novice so apologies for any wrong terminology. I've had a look on SE and I know how to recursively descend into a directory structure, however all the working examples I can find seem to pipe the output of Get-ChildItem, which I can't understand how to further work with the file before the loop iterates.
What I'm trying to do
Provide my script with a directory: C:\Users\Donglecow\myPictures\.
Recursively loop through all directories in the defined directory (I have an extensive file structure).
Find all files which match a filter thumbnail*.jpg, EG: thumbnail_001.jpg
Rename the file to BACKUP-thumbnail*.jpg
Check the dimensions of that image in pixels and print them out to the command line
What I've got so far
param (
[string]$dir = $pwd
)
write-output "Working from $($dir)"
$foundJpg = get-childitem -Recurse -Filter thumbnail*.jpg $dir
foreach($file in $foundJpg) {
write-output "Found $($file)"
#Rename to Backup...
#Check image dimensions...
#Do other stuff...
}
This works and outputs the correct file name, but when I try to rename the file I get an error.
Rename-Item -NewName { $_.Name.replace("thumbnail", "SAFE-thumbnail.jpg") }
Rename-Item : Cannot evaluate parameter 'NewName' because its argument is specified as a script block and there is no
input. A script block cannot be evaluated without input.
At C:\Users\Donglecow\myPictures\med.ps1:9 char:23
+ ... -Item -NewName { $_.Name.replace("thumbnail", "SAFE-thumbnail.jpg") }
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : MetadataError: (:) [Rename-Item], ParameterBindingException
+ FullyQualifiedErrorId : ScriptBlockArgumentNoInput,Microsoft.PowerShell.Commands.RenameItemCommand
I've deducted this is probably because I've tried to adapt from a command that pipes the output as shown in answers on this question:
Get-ChildItem -Recurse -Filter *.mp3 | Rename-Item –NewName { $_.name –replace 'xyz','abc' }
Whereas I'm using a different construct for my loop so that I can further interact with each file that I find.
My environment
For reference, a snippet of my file structure is below.
C:\Users\Donglecow\myPictures\
1\001\thumbnail_001.jpg
1\002\thumbnail_001.jpg
1\003\thumbnail_001.jpg
2\001\thumbnail_001.jpg
etc...
Where am I going wrong here?
Try this:
foreach($file in $foundJpg) {
$File | Rename-Item -NewName { $_.Name.replace("thumbnail", "SAFE-thumbnail.jpg") }
#Do other stuff...
}
You need to use $file as the input of the command (which you should be able to do via the pipeline as shown above) as this is what your ForEach loop is using to represent each item retrieved by Get-ChildItem into the $foundJpg variable.
I need to rename all files below pwd named all_v4_0.csv to all_v4_1.csv.
So far, I have worked my way to this piece of PowerShell:
$oldfiles = dir -recurse | ?{$_.Name -eq "all_v4_0.csv"}
foreach ($o in $oldfiles) {
$o.CopyTo Join-Path $o.Directory.ToString() "all_v4_1.csv"
}
But the foreach loop fails with the message that
At line:2 char:15
+ $o.CopyTo Join-Path $o.Directory.ToString() "all_v4_1.csv"
+ ~~~~~~~~~
Unexpected token 'Join-Path' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : UnexpectedToken
What am I doing wrong here?
Update, 20150604
As commented below by Manuel Batsching, the original version can be fixed by adding two layers of parentheses: one to indicate function argument, and one to force evaluation order:
$oldfiles = dir -recurse | ?{$_.Name -eq "all_v4_0.csv"}
foreach ($o in $oldfiles) {
$o.CopyTo((Join-Path $o.Directory.ToString() "all_v4_1.csv"))
}
For the problem at hand, one of the solutions with .FullName.Replace would probably be easier.
PSH's parser is not reading the Join-Path and its arguments as an expression to evaluate and pass result to outer expression. So parentheses to force evaluation order.
But additionally CopyTo is a .NET member, rather than a PSH cmdlet, so it needs to have parentheses around its argument (and no space).
Thus:
$o.CopyTo((Join-Path $o.Directory.ToString() "all_v4_1.csv"))
(Possibly using PSH's Copy-Item cmdlet would be a cleaner option.)
Set the target name separately before doing the copy. The below should work:
$oldfiles = dir -recurse | ?{$_.Name -eq "all_v4_0.csv"}
foreach ($o in $oldfiles) {
$newName = $o.FullName.replace("all_v4_0.csv","all_v4_1.csv")
Copy-Item $o.FullName $newName
}
If you want to keep things simple you can also use string concatenation to create the target path.
Get-ChildItem -Recurse -Include 'all_v4_0.csv' |
ForEach { $_.MoveTo($_.Directory.FullName + '\all_v4_1.csv') }
Simply replace the substring in the new name of the files:
Get-ChildItem -Recurse -Include 'all_v4_0.csv' |
Rename-Item -NewName { $_.Name.Replace('4_0', '4_1') }