Ignore wildcard characters [] in Powershell strings - powershell

I've got a Powershell script that takes a filename as input.
param([string]$folder, [string]$input_filename)
$logfile=$folder + "\duplicate.log"
Write-Output "`nScript: `n$transformation_filename`nLogfile: " | Out-file $logfile -Append
This works fine for most filenames, but it breaks down when the filename or folder name contains [].
dupfile.ps1 "C:\foldername [w]" "filename.xml"
Line 3 throws this error:
Cannot perform operation because the wildcard path C:foldername [w]\duplicate.log did not resolve to a file.
So the variables have to be treated more literally.
$logfile='$folder' + "\duplicate.log"
Nope, that's too literal.
$logfile="$folder" + "\duplicate.log"
Nope, same error.
How can I tell Powershell to insert the variable value but don't try to interpret anything in that value as wildcards?
Edit: I'm using Powershell 5.
My dupfile.ps1 contains only the 3 lines above.
Using this from the PS shell to run the script:
C:_SCRIPTS_\dupfile.ps1 "C:\foldername [w]" "test.txt"
One more addition, if I may. I have a few lines that redirect their error messages to the logfile.
Copy-Item -LiteralPath $inputPath -Destination $outputPath 2>&1|%{ "$_" } >>$logfile
The redirection trips over the [] again. I don't think I can use -LiteralPath here.

tl;dr
To ensure that file-path argument $var in redirection operations > $var / >> $var is treated literally if it contains [ and ], use
| Out-File -LiteralPath $var / | Out-File -LiteralPath $var -Append instead.
Copy-Item -LiteralPath $inputPath -Destination $outputPath 2>&1 |
ForEach-Object { "$_" } |
Out-File -LiteralPath $logfile # literal alternative to >> $logfile
As in your case, you may have to combine it with a redirection in order to ensure that additional streams beside the success output stream are targeted:
# Literal equivalent of *>> $logfile
... *>&1 | Out-File -LiteralPath $logfile -Append
Unfortunately, it gets tricky with capturing a given other stream only, as you then need to use the appropriate -*Variable common parameter, such as the common -ErrorVariable parameter:
# Literal equivalent of 2>> $logfile
... -ErrorVariable errs
$errs | Out-File -LiteralPath $logfile -Append
Caveat:
It is tempting to try to bypass the above solutions by escaping the literal path so that when it is interpreted as a wildcard, it is treated literally, using [WildcardPattern]::Escape().
However, this does not work as expected as of PowerShell 7.2.6, because the escaped form of the literal path is then used as the literal file path - see GitHub issue #9475 and also the discussion about whether > / >>, ... should treat their arguments as wildcards to begin with.
# !! Arguably SHOULD work, but doesn't as of 7.2.6:
# The file that is created is LITERALLY named for the ESCAPED form:
# `[1`].txt
'hi' > ([WildcardPattern]::Escape('[1].txt'))
Background information:
The -Path (-FilePath) parameter of PowerShell's provider cmdlets expects wildcard expressions for targeting file-system items by a name or path pattern. This parameter is implicitly targeted by the first positional argument, i.e. if you don't explicitly name the target parameter - e.g., Get-ChildItem foo is implicitly the same as Get-ChildItem -Path foo.
Surprisingly, this doesn't just apply to file-reading cmdlets (e.g., Get-ChildItem, Get-Content), but also to file-writing cmdlets (e.g., Set-Content, Out-File).
Note that the parameter named -Path is called -FilePath in Out-File for historical reasons. In PowerShell (Core) 7+, -Path was introduced as an alias name, for consistency.
Arguably, in file-writing/creating cmdlets this behavior is inappropriate - given that you usually know exactly what literal path you want to target - and there are several GitHub issues discussing this; an overview of all relevant discussions can be found in this comment on GitHub issue #17106.
In order for such arguments to be treated literally (verbatim), you must
use the -LiteralPath parameter instead.
In effect, > $var and >> $var are aliases for | Out-File -FilePath $var and | Out-File -FilePath $var -Append, which implies that $var is interpreted as a wildcard.
Using explicit Out-File calls with -LiteralPath, as shown at the top, avoids this problem.

Related

unable to read contents of file

I'm attempting to read the contents of a file:
$releaseNotesPath = "$(System.DefaultWorkingDirectory)\_ccp-develop\ccp\ccp\ReleaseNotes\ReleaseNotes\"
$latestReleaseNotesFile = Get-ChildItem -Path $releaseNotesPath -Filter *.txt | Select-Object FullName,Name | Sort-Object -Property Name | Select-Object -First 1
The issue occurs here:
$releaseNote = Get-Content $latestReleaseNotesFile
2021-11-14T14:29:07.0729088Z ##[error]Cannot find drive. A drive with the name '#{FullName=D' does not exist.
2021-11-14T14:29:07.1945879Z ##[error]PowerShell exited with code '1'.
What am I doing wrong?
You need to provide the file path (FullName):
$releaseNote = Get-Content $latestReleaseNotesFile.FullName
Shayki Abramczyk already answered how, I'll chime in with the why part.
So, let's see what goes on, step by step
# Assign a value to variable, simple enough
$latestReleaseNotesFile =
# Get a list of all
Get-ChildItem -Path $releaseNotesPath -Filter *.txt |
# Interested only on file full name and shortname. Here's the catch
Select-Object FullName,Name |
# Sort the results by name
Sort-Object -Property Name |
# Return the first object of collection.
Select-Object -First 1
Note that in the catch part, you are implicitly creating a new, custom Powershell object that contains two members: a fully qualified file name and short name. When you pass the custom object later to Get-Content, it doesn't know how to process the custom object. So, thus the error. Shayki's answer works, as it explicitly tells to use the FullName member that contains, well file's full name.
There's good information in the existing answers; let me summarize and complement them:
A simplified and robust reformulation of your command:
$latestReleaseNotesFile =
Get-ChildItem -LiteralPath $releaseNotesPath -Filter *.txt |
Select-Object -First 1
$releaseNote = $latestReleaseNotesFile | Get-Content
Get-ChildItem -LiteralPath parameter ensures that its argument is treated literally (verbatim), as opposed to as a wildcard expression, which is what -Path expects.
Get-ChildItem's output is already sorted by name (while this fact isn't officially documented, it is behavior that users have come to rely on, and it won't change).
By not using Select-Object FullName, Name to transform the System.IO.FileInfo instances output by Get-ChildItem to create [pscustomobject] instances with only the specified properties, the resulting object can as a whole be piped to Get-Content, where it is implicitly bound by its .PSPath property value to -LiteralPath (whose alias is -PSPath), which contains the full path (with a PowerShell provider prefix).
See this answer for details on how this pipeline-based binding works.
As for what you tried:
Get-Content $latestReleaseNotesFile
This positionally binds the value of variable $latestReleaseNotesFile to the Get-Content's -Path parameter.
Since -Path is [string[]]-typed (i.e., it accepts one or more strings; use Get-Help Get-Content to see that), $latestReleaseNotesFile's value is stringified via its .ToString() method, if necessary.
Select-Object FullName, Name
This creates [pscustomobject] instances with with .FullName and .Name properties, whose values are taken from the System.IO.FileInfo instances output by Get-ChildItem.
Stringifying a [pscustomobject] instance yields an informal, hashtable-like representation suitable only for the human observer; e.g.:
# -> '#{FullName=/path/to/foo; Name=foo})'
"$([pscustomobject] #{ FullName = '/path/to/foo'; Name = 'foo' }))"
Note: I'm using an expandable string ("...") to stringify, because calling .ToString() directly unexpectedly yields the empty string, due to a longstanding bug described in GitHub issue #6163.
Unsurprisingly, passing a string with content #{FullName=/path/to/foo; Name=foo}) is not a valid file-system path, and resulted in the error you saw.
Passing the .FullName property value instead, as shown in Shayki's answer, solves that problem:
For full robustness, it is preferable to use -LiteralPath instead of the (positionally implied) -Path
Specifically, paths that contain verbatim [ or ] will otherwise be misinterpreted as a wildcard expression.
Get-Content -LiteralPath $latestReleaseNotesFile.FullName
As shown at the top, sticking with System.IO.FileInfo instances and providing them via the pipeline implicitly binds robustly to -LiteralPath:
# Assumes that $latestReleaseNotesFile is of type [System.IO.FileInfo]
# This is the equivalent of:
# Get-Content -LiteralPath $latestReleaseNotesFile.PSPath
$latestReleaseNotesFile | Get-Content
Pitfall: One would therefore expect that passing the same type of object as an argument results in the same binding, but that is not true:
# !! NOT the same as:
# $latestReleaseNotesFile | Get-Content
# !! Instead, it is the same as:
# Get-Content -Path $latestReleaseNotesFile.ToString()
Get-Content $latestReleaseNotesFile
That is, the argument is not bound by its .PSPath property value to -LiteralPath; instead, the stringified value is bound to -Path.
In PowerShell (Core) 7+, this is typically not a problem, because System.IO.FileInfo (and System.IO.DirectoryInfo) instances consistently stringify to their full path (.FullName property value) - however, it still malfunctions for literal paths containing [ or ].
In Windows PowerShell, such instances situationally stringify to the file name (.Name) only, making malfunctioning and subtle bugs likely - see this answer.
This problematic asymmetry is discussed in GitHub issue #6057.
The following is a summary of the above with concrete guidance:
Robustly passing file-system paths to file-processing cmdlets:
Note: The following applies not just to Get-Content, but to all file-processing standard cmdlets - with the unfortunate exception of Import-Csv in Windows PowerShell, due to a bug.
as an argument:
Use -LiteralPath explicitly, because using -Path (which is also implied if neither parameter is named) interprets its argument as a wildcard expression, which notably causes literal file paths containing [ or ] to be misinterpreted.
# $pathString is assumed to be a string ([string])
# OK: -LiteralPath ensures interpretation as a literal path.
Get-Content -LiteralPath $pathString
# Same as:
# Get-Content -Path $pathString
# !! Path is treated as a *wildcard expression*.
# !! This will often not matter, but breaks with paths with [ or ]
Get-Content $pathString
Additionally, in Windows PowerShell, when passing a System.IO.FileInfo or System.IO.DirectoryInfo instance, explicitly use the .FullName (file-system-native path) or .PSPath property (includes a PowerShell provider prefix; path may be based on a PowerShell-specific drive) to ensure that its full path is used; this is no longer required in PowerShell (Core) 7+, where such instances consistently stringify to their .FullName property - see this answer.
# $fileSysInfo is assumed to be of type
# [System.IO.FileInfo] or [System.IO.DirectoryInfo].
# Required for robustness in *Windows PowerShell*, works in both editions.
Get-Content -LiteralPath $fileSysInfo.FullName
# Sufficient in *PowerShell (Core) 7+*:
Get-Content -LiteralPath $fileSysInfo
via the pipeline:
System.IO.FileInfo and System.IO.DirectoryInfo instances, such as emitted by Get-ChildItem and Get-Item, can be passed as a whole, and robustly bind to -LiteralPath via their .PSPath property values - in both PowerShell editions, so you can safely use this approach in cross-edition scripts.
# Same as:
# Get-Content -LiteralPath $fileSysInfo.PSPath
$fileSysInfo | Get-Content
This mechanism - explained in more detail in this answer - relies on a property name matching a parameter name, including the parameter's alias names. Therefore, input objects of any type that have either a .LiteralPath, a .PSPath, or, in PowerShell (Core) 7+ only, a .LP property (all alias names of the -LiteralPath parameter) are bound by that property's value.[1]
# Same as:
# Get-Content -LiteralPath C:\Windows\win.ini
[pscustomobject] #{ LiteralPath = 'C:\Windows\win.ini' } | Get-Content
By contrast, any object with a .Path property binds to the wildcard-supporting -Path parameter by that property's value.
# Same as:
# Get-Content -Path C:\Windows\win.ini
# !! Path is treated as a *wildcard expression*.
[pscustomobject] #{ Path = 'C:\Windows\win.ini' } | Get-ChildItem
Direct string input and the stringified representations of any other objects also bind to -Path.
# Same as:
# Get-Content -Path C:\Windows\win.ini
# !! Path is treated as a *wildcard expression*.
'C:\Windows\win.ini' | Get-Content
Pitfall: Therefore, feeding the lines of a text file via Get-Content to Get-ChildItem, for instance, can also malfunction with paths containing [ or ]. A simple workaround is to pass them as an argument to -LiteralPath:
Get-ChildItem -LiteralPath (Get-Content -LiteralPath Paths.txt)
[1] That this logic is only applied to pipeline input, and not also to input to the same parameter by argument is an unfortunate asymmetry discussed in GitHub issue #6057.

Reading a file path that contains wildcard characters

I'm working on a script that takes a directory name as input, and writes a log file to that directory.
This works in most cases:
param([string]$output_folder)
$logfile="$output_folder" + "\myscript.log"
Write-Output "logfile: " $logfile
(Get-Date -Format "[yyyy-MM-dd HH:mm:ss]") | Out-file $logfile -Append
except when the file path contains a wildcard:
E:\Publishing\User manual [en]\
This results in an error 'Cannot perform operation because the wildcard path E:\Publishing\User manual [en]\myscript.log did not resolve to a file.
I spent some time trying to sanitize the input, but I haven't found a way that works yet.
$logfile='$output_folder' + "\myscript.log"
This treats the string as a literal, but that prevents $output_folder from being expanded.
param([io.directoryinfo]$output_folder)
doesn't help either.
Edit:
A comment suggested this:
Out-file -LiteralPath $logfile -Append
That works for the instances where I use Out-file.
I also call an external program, and I can't use -LiteralPath here:
"C:\Program Files\Saxonica\SaxonHE9.8N\bin\Transform.exe" -s:"$inputFile" -xsl:"$output_folder\$transformation_filename" -o:"$outputFile" -t 1>>$logfile 2>&1
in fact I have more arguments here that need to be treated as literals. Is there a way so specify this when I create the string?
$logfile="$output_folder" + "\myscript.log"

Cannot find path _ because it does not exist

I'm trying to create small script in powershell that would move files and directories to correct localizations. I made the following command:
Get-ChildItem -Path '.\list\' | ForEach-Object { if ($($_.Name) -like '*[1]*') {
$file = $($_.Name)
$path = $($_.FullName)
echo "$file ==> $path"
Move-Item -Path $path -Destination .\[1]\}}
and it detects correct files and directories, but doesn't move them.
Then I decided to modify command a bit and create hardlinks instead:
Get-ChildItem -Path '.\list\' | ForEach-Object { if ($($_.Name) -like '*[1]*') {
$file = $($_.Name)
$path = $($_.FullName)
echo "$file ==> $path"
New-Item -Path ".\``[1``]\" -Name $file -Type HardLink -Target "$path"}}
and I received the following response (cut to only 1 loop):
[1] dir1 ==> D:\test_move\list\[1] dir1
New-Item:
Line |
5 | New-Item -Path ".\``[1``]\" -Name $file -Type HardLink -Target "$path …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Cannot find path 'D:\test_move\list\[1] dir1' because it does not exist.
The same error appears both with and without administrative privileges.
What do I have to do to make it work?
Try the following:
Get-ChildItem -LiteralPath .\list -File -Filter '*[1]*' | ForEach-Object {
$file = $_.Name
$path = $_.FullName
"$file ==> $path" # implicit `echo` aka `Write-Output`
New-Item -Force -Type HardLink `
-Path (Join-Path .\[1] $file) `
-Target ([WildcardPattern]::Escape($path)) ` # !! see PowerShell Core comments below
-WhatIf
}
Note: The -WhatIf common parameter in the command above previews the operation. Remove -WhatIf once you're sure the operation will do what you want.
-Filter '*[1]*' prefilters the Get-ChildItem output to only include files whose name contains substring [1] verbatim, because the -Filter parameter uses a filesystem-native wildcard language that does not treat [ and ] as metacharacters.
By contrast, with PowerShell's more powerful wildcard patterns, '*[1]*' would match any name that contains just 1, because the [...] is interpreted as a character set or range. With the -like, the wildcard matching operator operator, you'd have to use '*`[1`]*' (escaping of the metacharacters to be interpreted verbatim with `) to find verbatim [1] substrings.
-File limits matching items to files, because hardlinks are only supported for files, not also directories.
-Path (Join-Path .\[1] $file) use only a -Path argument - rather than a directory-path-only -Path argument combined with a filename-only -Name argument - which ensures that the argument is treated as a literal (verbatim) path, without interpretation of wildcard metacharacters such as [ and ].
Regrettably, combining -Path with -Name causes the -Path argument to be interpreted as a wildcard pattern.
-Force creates the target directory on demand, if needed, but note that it would also replace any preexisting target file.
Windows PowerShell: ([WildcardPattern]::Escape($path)) escapes the -Target (aka -Value) argument (target path) in order to treat it verbatim, because it is - unfortunately - interpreted as a wildcard pattern. Not performing this escaping prompted the error you saw.
Caveat:
In PowerShell [Core] 7+, a breaking change was approved in GitHub proposal #13136 to - more sensibly - treat the -Target argument as a literal (verbatim) path, in which case you would simply use -Target $path.
However, as of PowerShell 7.2.2 this change isn't implemented yet, and, unfortunately, targeting a path that contains [ and ] is currently broken altogether - see GitHub issue #14534.
As of PowerShell 7.2.2 you must escape twice(!) to make it work: ([WildcardPattern]::Escape([WildcardPattern]::Escape($path)))
Note that many, but not all, file-processing cmdlets offer a -LiteralPath parameter to explicitly pass paths to be taken literally (verbatim), whereas the -Path parameter - usually the implied parameter for the first positional argument - is designed to accept wildcard patterns.
Therefore, your could have made your original approach with Move-Item work as follows:
# Ensure that the target dir. exists.
# No escaping needed for -Path when not combined with -Name.
$null = New-Item -Type Directory -Path .\[1] -Force
# Move the file, targeted with -LiteralPath, there.
# No escaping needed for -Destination.
Move-Item -LiteralPath $path -Destination .\[1]\
Note: Unlike with New-Item, Move-Item's -Force does not create the target directory on demand. On the flip side, Move-Item's -Destination more sensibly interprets its argument literally (verbatim), unlike New-Item's -Target parameter.

Writing console output to a file - file is unexpectedly empty

I'm new to scripting and I am trying to write the information returned about a VM to a text file. My script looks like this:
Connect-VIServer -Server 192.168.255.255 -Protocol https -User xxxx -Password XXXXXX
Get-VM -Name xxxxxx
Get-VM xxxxx | Get-HardDisk | Select Parent, Name, Filename, DiskType, Persistence | FT -AutoSize
Out-File -FilePath C:Filepath
I am able to connect to the VM, retrieve the HDD info and see it in the console. The file is created where I want it and is correctly named. No data is ever put into the file. I have tried Tee-Object with the same results. I've also tried the -append switch. I did see a post about the data being returned as an array and Powershell is not able to move the data from an array to a string. Do I need to create a variable to hold the returned data and write to file from there?
Thanks
Guenther Schmitz' answer is effective, but it's worth explaining why:
Your Out-File -FilePath C:Filepath is a stand-alone command that receives no input.
An Out-File call with no input simply creates an empty file (0 bytes).
In order for cmdlets such as Out-File to receive input from (an)other command(s) (represented as ... below), you must use the pipeline, which means that you must place a | after the input command(s) and follow it with your Out-File call:Note that I'm using the shorter -Path parameter alias for the less commonly used -FilePath[1]
... | Out-File -Path C:Filepath
In the simplest case, as above, the entire command (pipeline) is placed on the same line; if you want to spread it across multiple lines for readability, you have have two choices:
Put a line break immediately after |, which tells PowerShell that the command continues on the next line:
... |
Out-File -Path C:Filepath
End a line with an explicit line continuation, which means placing ` at the very end of a line:
... `
| Out-File -Path C:Filepath
Alternatively, since you're using Out-File with its default behavior, you could use >, an output redirection, instead:
... > C:Filepath
A couple of asides:
Using Out-File with something other than strings, and using Format-* cmdlets in general, means that the output is only suitable for display (human consumption), not for further programmatic processing.
If you want to send output to both the console and a file, use the Tee-Object cmdlet, as TobyU suggests:
... | Tee-Object -Path C:Filepath
[1] Strictly speaking, -LiteralPath is the best choice in this case, because -Path interprets its arguments as wildcard expressions. However, omitting -Path, i.e. specifying the file path as a positional argument, as is common, implicitly binds to -Path.
try this:
Get-VM xxxxx |
Get-HardDisk |
Select Parent, Name, Filename, DiskType, Persistence |
Out-File -FilePath C:\Filepath

Powershell Get-Content failing spuriously

I have a fairly simple PS script that was working perfectly, and now has suddenly started giving errors. I have narrowed the problem portion to a couple of Get-Content statements. Here's what the affected part of the script looks like:
$pathSource = "D:\FileDirectory"
Set-Location -Path $pathSource
Get-Content -Encoding UTF8 -Path FilesA*.txt | Out-File -Encoding ASCII FilesA_Digest.txt
Get-Content -Encoding UTF8 -Path FilesB*.txt | Out-File -Encoding ASCII FilesB_Digest.txt
This part of the script gathers up a collection of like-named files and concatenates them into a single text file for uploading to an FTP site. The Get-Content/Out-File was needed as the original files are encoded incorrectly for the FTP site. The script was working perfectly, running once each night for several weeks. Now, it gets the following error when the Get-Content statements are reached:
Get-Content : A parameter cannot be found that matches parameter name 'Encoding'.
At D:\FileDirectory\Script.ps1
Environment is Windows Server 2016. I've tried different variations on the Get-Content parameters, but nothing has worked. I know there is a bug that affects network-mapped drives, but that's not the case here -- all files are local.
Any ideas/suggestions?
The only plausible explanation I can think of is that a custom Get-Content command that lacks an -Encoding parameter is shadowing (overriding) the standard Get-Content cmdlet in the PowerShell session that's executing your script.
To demonstrate:
# Define a custom Get-Content command (function) that accepts only
# a (positional) -Path parameter, not also -Encoding.
function Get-Content { [CmdletBinding()] param([string] $Path) }
# Now try to use Get-Content -Encoding
Get-Content -Encoding Utf8 FilesA*.txt
You'll see the same error message as in your question.
Use Get-Command Get-Content -All to see all commands named Get-Content, with the effective command listed first.
Then examine where any custom commands may come from; e.g., your $PROFILE script may contain one.
To rule out $PROFILE as the culprit, start PowerShell without loading the profile script and examine Get-Content then:
powershell -noprofile # Windows PowerShell
pwsh -noprofile # PowerShell Core
A simple way to rule out custom overrides ad hoc is to call a command by its module-qualified name:
Microsoft.Powershell.Management\Get-Content ...
You can determine a built-in cmdlet's module name of origin as follows:
PS> (Get-Command Get-Content -All)[-1].ModuleName
Microsoft.PowerShell.Management
In a pinch you can also infer the originating module name from the URL of the help topic:
Googling Get-Content will take you to https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-content - note how the cmdlet's module name, microsoft.powershell.management (case doesn't matter), is the penultimate (next to last) URI component.
It seems an issue with the out command. Can you please try below code :
$pathSource = "D:\FileDirectory"
Set-Location -Path $pathSource
Get-Content -Encoding UTF8 -Path FilesA*.txt | Set-Content -Encoding ASCII -path FilesA_Digest.txt
Get-Content -Encoding UTF8 -Path FilesB*.txt | Set-Content -Encoding ASCII -path FilesB_Digest.txt
Well, I don't know why it failed, but I can say that I have completely re-written the script and now it works. I have to note that, given the errors that were occurring, I also don't know why it is now working.
I am using the exact same calls to the Get-Content commandlet, with the -Encoding parameter and the pipe to Out-File with its own -Encoding parameter. I am doing the exact same actions as the previous version of the script. The only part that is significantly different is the portion that performs the FTP transfer of the processed files. I'm now using only PowerShell to perform the transfer rather than CuteFTP and it all seems to be working correctly.
Thanks to everyone who contributed.
Cheers
Norm
Not sure if it helps, but I was running into the same with:
$n = ni '[hi]' -value 'some text'
gc $n -Encoding Byte
$f = ls *hi*
$f.where{$_.name -eq '[hi]'}.Delete()
also looks like there's already a chain of SOs about this known bug see this answer