Script returning error: "Get-Content : An object at the specified path ... does not exist, or has been filtered by the -Include or -Exclude parameter - powershell

EDIT
I think I now know what the issue is - The copy numbers are not REALLY part of the filename. Therefore, when the array pulls it and then is used to get the match info, the file as it is in the array does not exist, only the file name with no copy number.
I tried writing a rename script but the same issue exists... only the few files I manually renamed (so they don't contain copy numbers) were renamed (successfully) by the script. All others are shown not to exist.
How can I get around this? I really do not want to manually work with 23000+ files. I am drawing a blank..
HELP PLEASE
I am trying to narrow down a folder full of emails (copies) with the same name "SCADA Alert.eml", "SCADA Alert[1].eml"...[23110], based on contents. And delete the emails from the folder that meet specific content criteria.
When I run it I keep getting the error in the subject line above. It only sees the first file and the rest it says do not exist...
The script reads through the folder, creates an array of names (does this correctly).
Then creates an variable, $email, and assigns the content of that file. for each $filename in the array.
(this is where is breaks)
Then is should match the specific string I am looking for to the content of the $email var and return true or false. If true I want it to remove the email, $filename, from the folder.
Thus narrowing down the email I have to review.
Any help here would be greatly appreciated.
This is what I have so far... (Folder is in the root of C:)
$array = Get-ChildItem -name -Path $FolderToRead #| Get-Content | Tee C:\Users\baudet\desktop\TargetFile.txt
Foreach ($FileName in $array){
$FileName # Check File
$email = Get-Content $FolderToRead\$FileName
$email # Check Content
$ContainsString = "False" # Set Var
$ContainsString # Verify Var
$ContainsString = %{$email -match "SYS$,ROC"} # Look for String
$ContainsString # Verify result of match
#if ($ContainsString -eq "True") {
#Remove-Item $FolderToRead\$element
#}
}

Here's a PowerShell-idiomatic solution that also resolves your original problems:
Get-ChildItem -File -LiteralPath $FolderToRead | Where-Object {
(Get-Content -Raw -LiteralPath $_.FullName) -match 'SYS\$,ROC'
} | Remove-Item -WhatIf
Note: The -WhatIf common parameter in the command above previews the operation. Remove -WhatIf once you're sure the operation will do what you want.
Note how the $ character in the RHS regex of the -match operator is \-escaped in order to use it verbatim (rather than as metacharacter $, the end-of-input anchor).
Also, given that $ is also used in PowerShell's string interpolation, it's better to use '...' strings (single-quoted, verbatim strings) to represent regexes, assuming no actual up-front string expansion is needed before the regex engine sees the resulting string - see this answer for more information.
As for what you tried:
The error message stemmed from the fact that Get-Content $FolderToRead\$FileName binds the file-name argument, $FolderToRead\$FileName, implicitly (positionally) to Get-Content's -Path parameter, which expects PowerShell wildcard patterns.
Since your file names literally contain [ and ] characters, they are misinterpreted by the (implied) -Path parameter, which can be avoided by using the -LiteralPath parameter instead (which must be specified explicitly, as a named argument).
%{$email -match "SYS$,ROC"} is unnecessarily wrapped in a ForEach-Object call (% is a built-in alias); while that doesn't do any harm in this case, it adds unnecessary overhead;
$email -match "SYS$,ROC" is enough, though it needs to be corrected to
$email -match 'SYS\$,ROC', as explained above.

[System.IO.Directory]::EnumerateFiles($Folder) |
Where-Object {$true -eq [System.IO.File]::ReadAllText($_, [System.Text.Encoding]::UTF8).Contains('SYS$,ROC') } |
ForEach-Object {
Write-Host "Removing $($_)"
#[System.IO.File]::Delete($_)
}
Your mistakes:
%{$email -match "SYS$,ROC"} - What % is intended to be? This is ForEach-Object alias.
%{$email -match "SYS$,ROC"} - Why use -match? This is much slower than -like or String.Contains()
%{$email -match "SYS$,ROC"} - When using $ inside double quotes, you should escape this using single backtick symbol (I have `$100). Otherwise, everything after $ is variable name: Hello, $username; I's $($weather.ToString()) today!
Write debug output in a right way: use Write-Debug, Write-Verbose, Write-Host, Write-Warning, Write-Error, Write-Information.
Can be better:
Avoid using Get-ChildItem, because Get-ChildItem returns files with attributes (like mtime, atime, ctime, etc). This additional info is additional request per file. When you need only list of files, use native .Net EnumerateFiles from System.IO.Directory. This is significant performace boost on huge amounts of files.
Use RealAllText or ReadAllLines or ReadAllBytes from System.IO.File static class to be more concrete instead of using universal Get-Content.
Use pipelines ;-)

Related

Extract words from filename delineated by underscores and spaces in Powershell

I am trying to extract two words from filenames. The names have the format:
__XXXXXXXX_XXX_XXXXXXX_XXXX_XXXXX_XXXX XXX_Aircraft 017_XXXXXXXX-XXXXXXX_XXXXXXX-XXXXXXX-XXXXXX-01Apr2021-XXXXX
With the X's being replaced with different words. I need to extract the aircraft number and the date so that I can rename the files with just that information. Using help from this site I have tried the following to isolate the aircraft number:
$names = gci -Path "H:\Path\to\Logs" *.log -Recurse | select #{n="Name"; e={if ($_.Name -match "Aircraft (\w+)") {
$matches[1] }}}
However, it doesn't seem to give me the match I need. However, I am very inexpert in programming and may be going down the wrong path. My hope is that the same logic used to isolate the aircraft number also applies for the date.
# Create a sample file.
$file = New-Item '__XXXXXXXX_XXX_XXXXXXX_XXXX_XXXXX_XXXX XXX_Aircraft 017_XXXXXXXX-XXXXXXX_XXXXXXX-XXXXXXX-XXXXXX-01Apr2021-XXXXX'
# Substitute your `Get-ChildItem` command for $file
$file |
Rename-Item -WhatIf -NewName {
if ($_.Name -match '_(Aircraft \w+?)_.+(\d{2}[a-z]{3}\d{4})-') {
# Synthesize the new file name from the extracted substrings.
'{0} - {1}' -f $Matches[1], $Matches[2]
} else {
# Input file name didn't match, (effectively) do nothing.
$_.Name
}
}
Note: The -WhatIf common parameter in the command above previews the operation. Remove -WhatIf once you're sure the operation will do what you want.
For an explanation of the regex used with the -match operator above, see this regex101.com page.[1]
The above uses two capture groups ((...)) to capture the substrings of interest, which can be accessed via indices 1 and 2 of the automatic $Matches variable.
-f, the format operator is then used to build the output file name from the captured substrings. Tweak the LHS format string as needed.
Thanks to -WhatIf, you'll see output such as the following, which is the preview of what would happen when you remove -WhatIf - note the new file name in the Destination: path:
What if: Performing the operation "Rename File" on target
"Item: /tmp/__XXXXXXXX_XXX_XXXXXXX_XXXX_XXXXX_XXXX XXX_Aircraft 017_XXXXXXXX-XXXXXXX_XXXXXXX-XXXXXXX-XXXXXX-01Apr2021-XXXXX
Destination: /tmp/Aircraft 017 - 01Apr2021".
Note how a script block ({ ... }) is passed as an argument to Rename-Item's -NewName parameter, which then acts on each input file via the automatic automatic $_ variable and outputs the argument value to use for the input object at hand. Such script blocks are called delay-bind script blocks.
[1] Note that even though regex101.com, a site for visualizing, explaining and experimenting with regexes, doesn't support the .NET regex engine used by PowerShell, choosing a similar engine, such as Java's, usually exhibits the same behavior, at least fundamentally.

How would I specify a directory to run a PowerShell script that would edit file extensions?

I am new to PowerShell and new to IT. I've been asked by my boss to write a PowerShell script that will identify filenames that have no file extension and then change them to .PDF files. After doing some research online I've found a script that had a similar purpose and tried to tailor it to my needs:
$proj_files = Get-ChildItem | Where-Object {$_.Extension -eq "."}
ForEach ($file in $proj_files) {
$filenew = $file.Name + ".pdf"
Rename-Item $file $filenew
}
My first question is does the logic in this script make sense? Is "Extension -eq "." the correct syntax to specify a filename with no extension? My other thought was to use Extension -eq "null" or something similar. If I do need to use a null value, what would that look like? My other question is how would I specify a given directory for this script to search through, or would I even need to? My thought here would be to specify the path under Get-ChildItem, like so: $proj_files = Get-ChildItem -Path C:\Users\mthomas\Documents | Where-Object {$_.Extension -eq ".'} Does that seem correct? I am hesitant to test this out before getting a second opinion because I don't want to change every file extension on my computer or something stupid like that. Anyhow, thanks everyone for the help.
You can do something like the following to find files in a directory without an extension, and rename them to have a PDF extension:
$directory = "C:\Path\To\Directory"
Get-ChildItem -File $directory | Where-Object { -Not $_.Extension } | Foreach-Object {
$_ | Rename-Item -NewName "$($_.Name).pdf"
}
Let's break this down
$directory = "C:\Path\To\Directory"
This is where we set the directory we want to locate files without extensions in. It doesn't have to be set as a static variable but since you are just getting your feet wet with Powershell this keeps it simple.
Get-ChildItem -File $directory
Get-ChildItem is the cmdlet which is used to list directory contents (also aliased to gci, ls, and dir). -File tells it to only list files, and $directory references the directory we want to search from, which we set above. Note that Get-ChildItem might behave differently depending on the provider (for example, you can also use Get-ChildItem on a registry key), but if you are working with a filesystem path you do not need to worry about additional providers for this case.
|
Passes the previous output down the pipeline. This is a common operator in Powershell, but basically you can string commands together using it. You can read more about the pipeline at https://learn.microsoft.com/en-us/powershell/scripting/getting-started/fundamental/understanding-the-windows-powershell-pipeline?view=powershell-6
Where-Object { -Not $_.Extension }
Where-Object evaluates a condition on one or more items, and filters out items that do not meet the condition. Since Get-ChildItem can return one or more files, we use the -Not operator in the ScriptBlock (denoted by {} and make sure that there is no extension on the file. $_, or $PSItem, is a special variable used by the pipeline, in this case $_ equals each item returned by Get-ChildItem. The Extension property exists on files returned by Get-ChildItem, and will be blank, or evaluated as $False. So filtering on -Not $_.Extension is the same as saying to only match objects that are missing a file extension. Where-Object can be read about in more detail here: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/where-object?view=powershell-6
Foreach-Object { SCRIPTBLOCK }
Similar to Where-Object, but runs code for each object in the pipeline rather than evaluating and filtering out objects which don't match a condition. In this case, we pipe the each file without an extension to Rename-Item, which I'll break down further below. More information on Foreach-Object can be read about here: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-6
$_ | Rename-Item -NewName "$($_.Name).pdf"
Rename the current file in the Foreach-Object block to the new name with .pdf appended. The "$( ... )" is called a sub-expression, which is a string interpolation technique that lets you run a command within a string, and make its output part of the string. You could achieve the same effect by doing $_ | Rename-Item -NewName ( $_.Name + ".pdf" ) which just adds a .pdf to the end of the current name.
Summary
The pipeline is a very powerful tool in Powershell, and is key to writing efficient and less bloated scripts. It might seem complex at first, but the more you use it the less daunting it will seem. I highly suggest reading the additional documentation I linked to above as it should help fill in any gaps I may have missed in my explanations above.
To simplify the breakdown above, the command does this, in this order: Gets all files in the specified directory, selects only the files that do not have an extension, then renames each file found without an extension to have a .pdf at the end.
The logic in the script - the overall shape - makes understandable sense, but is not right for it to work as you intend.
Testing on my computer here:
new-item -ItemType File -Name 'test'
get-item test | format-list *
get-item test | foreach { $_.extension; $_.Extension.length; $_.extension.GetType().name }
a file with no extension shows up with an empty string (blank content, length 0, type String, so your where-object { $_.Extension -eq "." } needs to be looking for "" instead of ".".
But:
Get-ChildItem | Where-Object { $_.Extension -eq '' }
shows me some folders as well, because they also have no extension in their name, so you might want Get-ChildItem -File to restrict it to just files.
how would I specify a given directory for this script to search through, or would I even need to?
It would run in the current directory, whichever shows up in your prompt PS C:\wherever> so if you need it to run somewhere else, yes you'd need to change to that folder or specify in get-childitem -LiteralPath 'c:\path\to\wherever'. You haven't mentioned subfolders, if you need those included, get-childitem -Recurse switch as well.
Speaking of subfolders, your $filenew = $file.Name + ".pdf" only makes sense in the current directory, I think it would work better if you used the full filename including path, so they definitely get renamed in the same place they were found $filenew = $file.FullName + ".pdf"
Is "Extension -eq "." the correct syntax to specify a filename with no extension?
Being careful here, what you wrote in your question was correct syntax but incorrect string content. What you've written here with quotes on the left of Extension is incorrect syntax.
My other thought was to use Extension -eq "null" or something similar. If I do need to use a null value, what would that look like?
And being careful here, "null" is not a null value, it's a string containing the four letter word 'null'.
You don't need to use a null value here, normally if you do it looks like $null, but in this case you could use where-object { [string]::IsNullOrEmpty($_.Extension) } but there's no benefit to it, I think.
And, as a stylistic choice, both "" and '' are strings, but "" can contain variables and sub-expressions, so if you have plain text it's a neat habit to use '' for it because it makes it clear to the reader that you intend there to be nothing special happening in this string.
Then your code, with parameter names given, looks more like:
$proj_files = Get-ChildItem -LiteralPath 'C:\Users\mthomas\Documents' |
Where-Object {$_.Extension -eq '.'}
foreach ($file in $proj_files)
{
$filenew = $file.FullName + '.pdf'
Rename-Item -LiteralPath $file.FullName -NewName $filenew
}
If you want to see what it will do, use -WhatIf on the end of Rename-Item:
Rename-Item -LiteralPath $file.FullName -NewName $filenew -WhatIf
Then it won't make the changes, just tell you what it would do.
I am hesitant to test this out before getting a second opinion because I don't want to change every file extension on my computer or something stupid like that
Sensible. But internet people are going to tell you to test their code before running it, because ultimately it's your responsibility to safeguard your files, rather than trust random code from the internet, so having test folders, having a spare machine, having a good backup, playing with PowerShell in pieces until you are happy with what they do, they're all good habits to get into as well.

In function repeat an action for each entered parameter

My main script run once gci on a specified drive via -path parameter , then it does multiple different tables from this output. Here below is a part of my script which does a specific table from an directory specified via -folder parameter, for example :
my-globalfunction -path d:\ -folder d:\folder
It work fine, but only for one entered folder path, the goal of this script is that user can enter multiple folders path and get a tables for each entered -folder parameter value, like this :
This clause in your Where-Object would be the issue:
$_.FullName.StartsWith($folder, [System.StringComparison]::OrdinalIgnoreCase)
The array of folders passed are most likely being cast as one long string which would never match. I had a regex solution posted but remembered a simpler way after looking at what your logic was trying to do.
Simpler Way
Even easier way is to put this information right into Get-ChildItem since it accepts string arrays for -Path. This way I don't think you even need to have 2 parameters since you never again use the results from $fol anyway. Based on the assumption that you were looking for all subfolders of $folder
$gdfolders = Get-ChildItem -Path $folder -Recurse -Force | Where-Object{$_.psiscontainer}
That would return all subfolders of the paths provided. If you have PowerShell 3.0 or higher this would even be easier.
$gdfolders = Get-ChildItem -Path $folder -Recurse -Force -Directory
Update from comments
The code you have displayed is incomplete which is what lead me to the solution that you see above. If you do use the variable $fol somewhere else that you do not show lets go back to my earlier regex solution which would work better in place with what you already have.
$regex = "^($(($folder | ForEach-Object{[regex]::Escape($_)}) -join "|")).+"
....
$gdfolders = $fol | Where-Object{($_.Attributes -eq "Directory") -and ($_.FullName -match $regex)}
What this will do is build a regex compare string with what I will assume is the logic of locate folders that begin with either of paths passed.
Using your example input of "d:\folder1", "d:\folder2" the variable $regex would work out to ^(d:\\folder1|d:\\folder2). The proper characters, like \, are escaped automatically by the static method [regex]::Escape which is applied to each element. We then use -join to place a pipe which, in this regex capture group means match whats on the left OR on the right. For completeness sake we state that the match has to occur at the beginning of the path with the caret ^ although this is most likely redundant. It would match paths that start with either "d:\folder1" or "d:\folder2". At the end of the regex string we have .+ which means match 1 to more characters. This should ensure we dont match the actual folder "d:\folder1" but meerly its children
Side Note
The quotes in the line with ’Size (MB)’ are not the proper ones which are '. If you have issues around that code consider changing the quotes.

Powershell - Find out if there are two files with the same name

I'm trying to check a series of directories and subdirectories to see if there are two or more files with the same name in the same folder. I think my issue is how I'm trying to validate a string.
For the following test directory:
|-- C:\Test\
|--C:\Test\YY\
|--C:\Test\YY\xx.log
|--C:\Test\YY\xx.txt
|-- C:\Test\ZZ\
|-- C:\Test\ZZ.log
|-- C:\Test\ZZ.txt
I need my code to find xx.log and ZZ.log. I can include a check against $_.PSisContainer (but I didn't think it was necessary).
ForEach ($item in (gci "C:\Test\*" -recurse)) {
If ($item.extension -notmatch "txt" -AND $item.basename+".txt" -eq $True) {
Write-Host $item.fullname
}
}
$item.basename+".txt" provides the right string but I can't use that string to validate the existence of the file.
Can anyone help correct my code? I'd like to learn how to handle concatenated strings like this--it's a trick I think would be useful in other areas.
This not intended as an answer, rather a comment, but SO is not allowing me to add comments right now :(
R_C_III - I guess there are two errors in the original code.
Firstly, it does not build the complete file path (i.e. DirectoryName + BaseName + .TXT). Rather than concatenate strings to do this, Keith has used PowerShell's ability to perform value substitutions inside double-quoted strings. e.g.
$s = "there"
write-host "hello $s"
results in
hello there
To insert object properties into a string in this way it's necessary to use sub-expressions as Keith explains.
$file = get-item "c:\somefile.txt"
write-host "$($file.DirectoryName)\$($file.BaseName).NEW"
results in
c:\somefile.NEW
Secondly (as per my comment on Keith's answer), the original code does not check for the existence of the '.txt' version of the file. The second clause in the IF statement attempts to equate the modified filename with $True ($item.basename+".txt" -eq $True). This clause will always evaluate as false.
Keith's modification adds the 'Test-Path' CmdLet which, in this instance, checks for the existence of the '.txt' file.
Try this:
gci C:\Test -recurse -file | Where {$_.extension -ne ".txt" -AND (Test-Path "$($_.DirectoryName)\$($_.BaseName).txt")}
You don't need the initial Foreach statement as PowerShell commands that output data can be used directly at the start of a pipeline. Then you filter pipeline objects with the Where command.

Error with Get-Content

I have the following portion of code:
$fileList | foreach {
Write-Host ". . . adding $_ to script"
$myCreateScript += Get-Content "$ScriptRoot\$_" | Out-String }
That produces the following toward the end of execution:
ItemNotFound,Microsoft.PowerShell.Commands.GetContentCommand
System.Exception: An object at the specified path E:\Scripts\users[[FILE_NAME_LOWER]]_administrator.sql does not exist, or has been filtered by the -Include or -Exclude parameter.
[[FILE_NAME_LOWER]]_administrator.sql is the name of the file, and it does exist in the specified directory. If this is a problem with the fact that the filename contains brackets ([]), I need to know how to resolve it within the foreach block, as I cannot easily change the filenames (they are used elsewhere in code that is in source control) and because of prior code in the script, need to be preserved in $fileList. I suspect the brackets are the issue, because $fileList has over 200 rows, and $myCreateScript gets populated with the correct data up to this point.
You need to use the -LiteralPath parameter of Get-Content:
$myCreateScript += Get-Content -LiteralPath "$ScriptRoot\$_" | Out-String }
Otherwise, [[FILE_NAME_LOWER]] will be interpreted as a wildcard character set which tells PowerShell to match any of the characters enclosed by the two square brackets on each end.