I've got a huge directory listing of files, and I need to see what special characters exist in the file names - specifically nonstandard characters like you'd get using ALT codes.
I can export a directory listing to a file easily enough with:
get-childitem -path D:\files\ -File -Recurse >output.txt
What I need to do however, is pull out the special characters, and only the special characters from the text file. The only way I can think to easily quantify everything "special" (since there are a ton of possibilities in the that character set) would be to compare the text against a list of characters I'd want to keep, stored in a joined variable (a-z, 0-9, etc)
I can't quite figure out how to pull out the "good" characters, leaving only the special ones. Any ideas on where to start?
I take "special" characters to be anything that falls outside US ASCII.
That basically means any character with a numerical value of 128 or more, easy to inspect in a Where-Object filter:
Get-ChildItem -File -Recurse |Where-Object {
$_.Name.ToCharArray() -gt 127
}
This will return all files containing "special" characters in their name.
If you want to extract the special characters themselves, per file, use ForEach-Object:
Get-ChildItem -File -Recurse |ForEach-Object {
if(($Specials = $_.Name.ToCharArray() -gt 127)){
New-Object psobject -Property #{File=$_.FullName;Specials=$(-join $Specials)}
}
}
Look at piping your results to Select-String. With Select-String you can specify a list of regex values to search for.
Related
folder name is: c:\home\alltext\
inside has: 2 text files with different names(each text contents extra whitespace that I want to trim)
text1.txt
text2.txt
I don't want to use notepad++ and do one by one text.txt if I have more than 2 command.
I tried PowerShell it returns both text1 and text2 together in same one text.txt.
How can I trim them in one command and return individual txt?
This is my command:
(get-content c:\home\alltext\*.txt).trim() -ne '' | Set-content c:\home\alltext\*.txt
You need to process the input files one by one:
Get-ChildItem c:\home\alltext*.txt | ForEach-Object {
Set-Content -LiteralPath $_.FullName -Value (($_ | Get-Content).Trim() -ne '')
}
Note that PowerShell never preserves the original character encoding when reading text files, so you may have to use the -Encoding parameter with Set-Content.
As for what you tried:
(get-content c:\home\alltext*.txt).trim() -ne '' streams the non-blank lines of all files matching wildcard expression c:\home\alltext*.txt, across file boundaries.
Perhaps surprisingly, not only does Set-Content's (positionally implied) -Path parameter accept wildcard expressions too, it writes the same content (the stringified versions of whatever input it receives) to whatever files happen to match that wildcard expression.
This problematic behavior is discussed in GitHub issue #6729; unfortunately, it was decided to retain the current behavior.
EDIT
I think I now know what the issue is - The copy numbers are not REALLY part of the filename. Therefore, when the array pulls it and then is used to get the match info, the file as it is in the array does not exist, only the file name with no copy number.
I tried writing a rename script but the same issue exists... only the few files I manually renamed (so they don't contain copy numbers) were renamed (successfully) by the script. All others are shown not to exist.
How can I get around this? I really do not want to manually work with 23000+ files. I am drawing a blank..
HELP PLEASE
I am trying to narrow down a folder full of emails (copies) with the same name "SCADA Alert.eml", "SCADA Alert[1].eml"...[23110], based on contents. And delete the emails from the folder that meet specific content criteria.
When I run it I keep getting the error in the subject line above. It only sees the first file and the rest it says do not exist...
The script reads through the folder, creates an array of names (does this correctly).
Then creates an variable, $email, and assigns the content of that file. for each $filename in the array.
(this is where is breaks)
Then is should match the specific string I am looking for to the content of the $email var and return true or false. If true I want it to remove the email, $filename, from the folder.
Thus narrowing down the email I have to review.
Any help here would be greatly appreciated.
This is what I have so far... (Folder is in the root of C:)
$array = Get-ChildItem -name -Path $FolderToRead #| Get-Content | Tee C:\Users\baudet\desktop\TargetFile.txt
Foreach ($FileName in $array){
$FileName # Check File
$email = Get-Content $FolderToRead\$FileName
$email # Check Content
$ContainsString = "False" # Set Var
$ContainsString # Verify Var
$ContainsString = %{$email -match "SYS$,ROC"} # Look for String
$ContainsString # Verify result of match
#if ($ContainsString -eq "True") {
#Remove-Item $FolderToRead\$element
#}
}
Here's a PowerShell-idiomatic solution that also resolves your original problems:
Get-ChildItem -File -LiteralPath $FolderToRead | Where-Object {
(Get-Content -Raw -LiteralPath $_.FullName) -match 'SYS\$,ROC'
} | Remove-Item -WhatIf
Note: The -WhatIf common parameter in the command above previews the operation. Remove -WhatIf once you're sure the operation will do what you want.
Note how the $ character in the RHS regex of the -match operator is \-escaped in order to use it verbatim (rather than as metacharacter $, the end-of-input anchor).
Also, given that $ is also used in PowerShell's string interpolation, it's better to use '...' strings (single-quoted, verbatim strings) to represent regexes, assuming no actual up-front string expansion is needed before the regex engine sees the resulting string - see this answer for more information.
As for what you tried:
The error message stemmed from the fact that Get-Content $FolderToRead\$FileName binds the file-name argument, $FolderToRead\$FileName, implicitly (positionally) to Get-Content's -Path parameter, which expects PowerShell wildcard patterns.
Since your file names literally contain [ and ] characters, they are misinterpreted by the (implied) -Path parameter, which can be avoided by using the -LiteralPath parameter instead (which must be specified explicitly, as a named argument).
%{$email -match "SYS$,ROC"} is unnecessarily wrapped in a ForEach-Object call (% is a built-in alias); while that doesn't do any harm in this case, it adds unnecessary overhead;
$email -match "SYS$,ROC" is enough, though it needs to be corrected to
$email -match 'SYS\$,ROC', as explained above.
[System.IO.Directory]::EnumerateFiles($Folder) |
Where-Object {$true -eq [System.IO.File]::ReadAllText($_, [System.Text.Encoding]::UTF8).Contains('SYS$,ROC') } |
ForEach-Object {
Write-Host "Removing $($_)"
#[System.IO.File]::Delete($_)
}
Your mistakes:
%{$email -match "SYS$,ROC"} - What % is intended to be? This is ForEach-Object alias.
%{$email -match "SYS$,ROC"} - Why use -match? This is much slower than -like or String.Contains()
%{$email -match "SYS$,ROC"} - When using $ inside double quotes, you should escape this using single backtick symbol (I have `$100). Otherwise, everything after $ is variable name: Hello, $username; I's $($weather.ToString()) today!
Write debug output in a right way: use Write-Debug, Write-Verbose, Write-Host, Write-Warning, Write-Error, Write-Information.
Can be better:
Avoid using Get-ChildItem, because Get-ChildItem returns files with attributes (like mtime, atime, ctime, etc). This additional info is additional request per file. When you need only list of files, use native .Net EnumerateFiles from System.IO.Directory. This is significant performace boost on huge amounts of files.
Use RealAllText or ReadAllLines or ReadAllBytes from System.IO.File static class to be more concrete instead of using universal Get-Content.
Use pipelines ;-)
I have a folder that contains files like 'goodthing 2007adsdfff.pdf', 'betterthing 2007adfdsw.pdf', and 'bestthing_2007fdsfad.pdf', I want to be able to rename each, eliminating all text including 2007 OR _2007 to the end of the string keeping .pdf and getting this result: 'goodthing.pdf' 'betterthing.pdf' 'bestthing.pdf' I've tried this with the "_2007", but haven't figured out a conditional to also handle the "2007". Any advice on how to accomplish this is greatly appreciated.
Get-ChildItem 'C:Temp\' -Name -Filter *.pdf | foreach { $_.Split("_2017")[0].substring(0)}
Try the following:
Get-ChildItem 'C:\Temp' -Name -Filter *.pdf |
Rename-Item -NewName { $_.Name -replace '[_ ][^.]+' } -WhatIf
Note: The -WhatIf common parameter in the command above previews the operation. Remove -WhatIf once you're sure the operation will do what you want.
The above uses Rename-Item with a delay-bind script block and the -replace operator as follows:
Regex [_ ][^.]+ matches everything from the first space or _ char. (character set [ _]) through to the following literal . char. ([^.]+ matches one or more chars. other than (^) than .) - that is, everything from the first / _ through to the filename extension (excluding the .).
Note: To guard against file names such as _2017.pdf matching (which would result in just .pdf as the new name), use the following regex instead: '(?<=.)[_ ][^.]+'
By not providing a replacement operand to -replace, what is matched is replace with the empty string and therefore effectively removed.
The net effect is that input files named
'goodthing 2007adsdfff.pdf', 'betterthing 2007adfdsw.pdf', 'bestthing_2007fdsfad.pdf'
are renamed to
'goodthing.pdf', 'betterthing.pdf', 'bestthing.pdf'
Without knowing the names of all the potential files, I can offer this solution that is 100%:
PS> $flist = ("goodthing 2007adsdfff.pdf","betterthing 2007adfdsw.pdf","bestthing_2007fdsfad.pdf")
PS> foreach ($f in $flist) {$nicename = ($f -replace "([\w\s]+)2007.*(\.\w+)", '$1$2') -replace "[\s_].","." ;$nicename}
goodthing.pdf
betterthing.pdf
bestthing.pdf
Two challenges:
the underscore is actually part of the \w character class. So the alternative to the above is to complicate the regex or try to assume that there will always be only one '_' before the 2007. Both seemed risky to me.
if there are spaces in filenames, there is no telling if you might encounter more than one. This solution removes only the one right before 2007.
The magic:
The -replace operator enables you to quickly capture text in () and re-use it in variables like $1$2. If you have more complex captures, you just have to figure out the order they are assigned.
Hope this helps.
I have various file names that are categorized in two different ways. They either just have a code like: "866655" or contain a suffix and prefix "eu_866655_001". My hope is to write to a text file the names of files in the same format. I cannot figure out a successful method for removing the suffix and prefix.
Currently this what I have in my loop in Powershell:
$docs = Get-ChildItem -Path $source | Where-Object {$_.Name -match '.doc*'}
if ($docs.basename -contians 'eu_*')
{
Write-Output ([io.fileinfo]"$doc").basename.split("_")
}
I'm hoping to turn "eu_866655_001" into "866655" by using "_" as the delimiter.
I'm aware that the answer is staring me down but I still can't seem to figure it out.
You could do something like the following. Feel free to tweak the -Filter on the Get-ChildItem command.
$source = 'c:\path\*'
$docs = Get-ChildItem -Path $source -File -Filter "*_*_*" -Include '*.doc','*.docx'
$docs | Rename-Item -NewName { "{0}{1}" -f $_.Basename.Split('_')[1],$_.Extension }
The important things to remember is that in order to use the -Include switch, you need an * at the end of the -Path value.
Explanation:
-Filter allows us to filter on names that contain two underscores separating three substrings.
-Include allows us to only list files ending in extensions .docx and .doc.
Rename-Item -NewName supports delayed script binding. This allows us use a scriptblock to perform any necessary operations for each piped object (each file).
Since the target files will always have two underscores, the .Split('_') method will result in an three index array delimited by the _. You have specified that you always want the second delimited substring and that is represented by index 1 ([1]).
The format operator (-f) puts the substring and extension together, completing the file name.
My main script run once gci on a specified drive via -path parameter , then it does multiple different tables from this output. Here below is a part of my script which does a specific table from an directory specified via -folder parameter, for example :
my-globalfunction -path d:\ -folder d:\folder
It work fine, but only for one entered folder path, the goal of this script is that user can enter multiple folders path and get a tables for each entered -folder parameter value, like this :
This clause in your Where-Object would be the issue:
$_.FullName.StartsWith($folder, [System.StringComparison]::OrdinalIgnoreCase)
The array of folders passed are most likely being cast as one long string which would never match. I had a regex solution posted but remembered a simpler way after looking at what your logic was trying to do.
Simpler Way
Even easier way is to put this information right into Get-ChildItem since it accepts string arrays for -Path. This way I don't think you even need to have 2 parameters since you never again use the results from $fol anyway. Based on the assumption that you were looking for all subfolders of $folder
$gdfolders = Get-ChildItem -Path $folder -Recurse -Force | Where-Object{$_.psiscontainer}
That would return all subfolders of the paths provided. If you have PowerShell 3.0 or higher this would even be easier.
$gdfolders = Get-ChildItem -Path $folder -Recurse -Force -Directory
Update from comments
The code you have displayed is incomplete which is what lead me to the solution that you see above. If you do use the variable $fol somewhere else that you do not show lets go back to my earlier regex solution which would work better in place with what you already have.
$regex = "^($(($folder | ForEach-Object{[regex]::Escape($_)}) -join "|")).+"
....
$gdfolders = $fol | Where-Object{($_.Attributes -eq "Directory") -and ($_.FullName -match $regex)}
What this will do is build a regex compare string with what I will assume is the logic of locate folders that begin with either of paths passed.
Using your example input of "d:\folder1", "d:\folder2" the variable $regex would work out to ^(d:\\folder1|d:\\folder2). The proper characters, like \, are escaped automatically by the static method [regex]::Escape which is applied to each element. We then use -join to place a pipe which, in this regex capture group means match whats on the left OR on the right. For completeness sake we state that the match has to occur at the beginning of the path with the caret ^ although this is most likely redundant. It would match paths that start with either "d:\folder1" or "d:\folder2". At the end of the regex string we have .+ which means match 1 to more characters. This should ensure we dont match the actual folder "d:\folder1" but meerly its children
Side Note
The quotes in the line with ’Size (MB)’ are not the proper ones which are '. If you have issues around that code consider changing the quotes.