When creating a new VM in Hyper-V, to keep things organized, I use a particular naming convention when creating the associated VHDX files. The naming convention is the VMs FQDN followed by the SCSI controller attachment point followed by what the name of the drive is called or used for inside of the VM. I encapsulate the SCSI and Name parameters inside smooth and square brackets respectively. I find this tends to make things a little bit easier from a human perspective to match the VHDX files in Hyper-V to what the VM sees internally when needing to do maintenance tasks. It has also helped with scripting in the past. An example file name would look as follows...
servername.example.com(0-0)[OS].vhdx
This has worked well for quite some time, but recently I tried to run some PowerShell commands against the VHDX files and ran across a problem. Apparently the square brackets for the internal VM name are being parsed as RegEx or something inside of the PowerShell commandlet (I'm honestly just guessing on this). When I try to use Get-VHD on a file with the above naming convention it spits out an error as follows:
Get-VHD : 'E:\Hyper-V\servername.example.com\Virtual Hard Disks\servername.example.com(0-0)[OS].vhdx' is not an existing virtual hard disk file.
At line:1 char:12
+ $VhdPath | Get-VHD
+ ~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Get-VHD], VirtualizationException
+ FullyQualifiedErrorId : InvalidParameter,Microsoft.Vhd.PowerShell.Cmdlets.GetVHD
If I simply rename the VHDX file to exclude the "[OS]" portion of the naming convention the command works properly. The smooth brackets for the SCSI attachment point don't seem to bother it. I've tried doing a replace command to add a backtick ''`'' in front of the brackets to escape them, but the same error results. I've also tried double backticks to see if passing in a backtick helped... that at least showed a single backtick in the error it spat out. Suspecting RegEx, I tried the backslash as an escape character too... which had the interesting effect of converting all the backslashes in the file path into double backslashes in the error message. I tried defining the path variable via single and double quotes without success. I've also tried a couple of different ways of obtaining it via pipeline such as this example...
((Get-VM $ComputerName).HardDrives | Select -First 1).Path | Get-VHD
And, for what it's worth, as many VMs as I am attempting to process... I need to be able to run this via pipeline or some other automation scriptable method rather than hand coding a reference to each VHDX file.
Still thinking it may be something with RegEx, I attempted to escape the variable string with the following to no avail:
$VhdPathEscaped = [System.Text.RegularExpressions.Regex]::Escape($VhdPath)
Quite frankly, I'm out of ideas.
When I first ran across this problem was when I tried to compact a VHDX file with PowerShell. But, since the single VM I was working with needed to be offline for that function to run anyway, rather than fight the error with the VHDX name, I simply renamed it, compacted it, and reset the name back. However, for the work I'm trying to do now, I can't afford to take the VM offline as this script is going to run against a whole fleet of live VMs. So, I need to know how to properly escape those characters so the Get-VHD commandlet will accept those file names.
tl;dr:
A design limitation of Get-VHD prevents it from properly recognizing VHD paths that contain [ and ] (see bottom section for details).
Workaround: Use short (8.3) file paths assuming the file-system supports them:
$fso = New-Object -ComObject Scripting.FileSystemObject
$VhdPath |
ForEach-Object { $fso.GetFile((Convert-Path -LiteralPath $_)) } |
Get-VHD
Otherwise, your only options are (as you report, in your case the VHDs are located on a ReFS file-system, which does not support short names):
Rename your files (and folders, if applicable) to not contain [ or ].
Alternatively, if you can assume that your VHDs are attached to VMs, you can provide the VM(s) to which the VHD(s) of interests are attached as input to Get-VHD, via Get-VM (you may have to filter the output down to only the VHDs of interest):
(Get-VM $vmName).Id | Get-VHD
Background information:
It looks like Get-VHD only has a -Path parameter, not also a -LiteralPath parameter, which looks like a design flaw:
Having both parameters is customary for file-processing cmdlets (e.g. Get-ChildItem):
-Path accepts wildcard expressions to match potentially multiple files by a pattern.
-LiteralPath is used to pass literal (verbatim) paths, to be used as-is.
What you have is a literal path that happens to look like a wildcard expression, due to use of metacharacters [ and ]. In wildcard contexts, these metacharacter must normally be escaped - as `[ and `] - in order to be treated as literals, which the following (regex-based) -replace operation ensures[1] (even with arrays as input).
Unfortunately, this appears not to be enough for Get-VHD. (Though you can verify that it works in principle by piping to Get-Item instead, which also binds to -Path).
Even double `-escaping (-replace '[][]', '``$&') doesn't work (which is - unexpectedly required in come cases - see GitHub issue #7999).
# !! SHOULD work, but DOES NOT
# !! Ditto for -replace '[][]', '``$&'
$VhdPath -replace '[][]', '`$&' | Get-VHD
Note: Normally, a robust way to ensure that a cmdlet's -LiteralPath parameter is bound by pipeline input is to pipe the output from Get-ChildItem or Get-Item to it.[2] Given that Get-VHD lacks -LiteralPath, this is not an option, however:
# !! DOES NOT HELP, because Get-VHD has no -LiteralPath parameter.
Get-Item -LiteralPath $VhdPath | Get-VHD
[1] See this regex101.com page for an explanation of the regex ($0 is an alias of $& and refers to the text captured by the match at hand, i.e. either [ or ]). Alternatively, you could pass all paths individually to the [WildcardPattern]::Escape() method (e.g., [WildcardPattern]::Escape('a[0].txt') yields a`[0`].txt.
[2] See this answer for the specifics of how this binding, which happens via the provider-supplied .PSPath property, works.
Ok... So, I couldn't get the escape characters to be accepted by Get-VHD... be it by hand or programmatically. I gave it a go of passing it on the pipeline using Get-ChildItem too without success. However... I did manage to find an alternative for my particular use case. In addition to a path to a VHDX file, the Get-VHD command will also accept vmid, and disknumber as parameters. So, not that it's the way I wanted to go about obtaining what I need (because this method spits out info on all the attached drives), I can still manage to accomplish the task at hand by using the following example:
Get-VM $ComputerName | Select-Object -Property VMId | Get-VHD
By referencing them in this manner the Get-VHD commandlet is happy. This works for today's problem only because the VHDX files in question are attached to VMs. However, I'll still need to figure out about referencing unattached files at some point in the future. Which... Maybe ultimately require a slow and painful renaming of all the VHDX files to not use the square brackets in their name.
Related
I was using tab autocompletion for a variable name in Powershell 5.1 today and noticed that one of the choices was the name of a PSDrive. The drive name is docs and I wanted to expand is called $document_name. When I typed $do<tab>, the shell did indeed expand what I had typed to $document_name but for some reason, I typed <tab> a second time and that's when the expanded text changed to $docs:.
I explored further and found that this type of variable exists for each of my PSDrives, or at least tab expansion suggests that it does.
More formally, for every PSDrive PSD, tab expansion believes that $PSD: is a valid thing.
My question is simple: what the heck are these? Here are some observations I've made so far:
These names are prefixed with $, so they look like PS variables. For the rest of this discussion (and in the earlier discussion above), I will assume they are variables and refer to them as such.
Although they appear to be variables, they are not listed in the Variable: PSDrive like most variables. In this way, it behaves like the $env "variable," which also is not listed in Variable:. I have a feeling if I could find documentation about $env, then I'd understand these objects also.
In some ways, they behave like pointers to filesystem objects. For example, if there is a file name readme.txt containing the text "Hello, world!" on a PSDrive named code, then all of the following are possible interactions with Powershell.
Fetch the contents of the file.
λ ${code:\readme.txt}
Hello, world!
Just to prove that the type of the above result is String:
λ ${code:\readme.txt} | % { $_.GetType().Name }
String
Trying to use this as a reference to the PSDrive doesn't work well for many operations, such as cd:
C:\
λ cd ${code:}
At line:1 char:4
+ cd ${code:}
+ ~~~~~~~~
Variable reference is not valid. The variable name is missing.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : InvalidBracedVariableReference
I could go on, but I'm stumped. If I pass $code: (or $env:, for that matter) to Get-Member, I get an error saying Variable reference is not valid.
So just what the heck are "variables" like $env and $<PSDrive>: (such as $code:)? Are they expressions? Built-in expressions? Some kind of object? Thanks for any help.
What you're seeing is namespace variable notation, which is a variable-based way to access the content of items in PowerShell drives whose underlying provider implements content-based access (i.e., implements the IContentCmdletProvider interface).
Terminology and documentation note:
As of this writing, the docs briefly explain namespace variable notation in the conceptual about_Scopes help topic, albeit without using that term and, somewhat confusingly, discuss it in the context of scope modifiers; while namespace qualifiers (such as $env:) are unrelated to scope modifiers (such as $script:), they use the same basic syntax form, however.[1]
The general syntax is:
${<drive>:<path>} # same as: Get-Content <drive>:<path>
${<drive>:<path>} = ... # same as: Set-Content <drive>:<path> -Value ...
The enclosing {...} aren't necessary if both the <drive> name and the <path> can syntactically serve as a variable name; e.g.:
$env:HOME # no {...} needed
${env:ProgramFiles(x86)} # {...} needed due to "(" and ")"
In practice, as of Windows PowerShell v5.1, the following in-box drive providers support namespace variable notation:
Environment (drive Env:)
Function (drive Function:)
Alias (drive Alias:)
FileSystem (drives C:, ...)
Variable (drive Variable:) - though virtually pointless, given that omitting the drive part accesses variables by default (e.g., $variable:HOME is the same as just $HOME).
Of these, the Env: drive is by far the most frequently used with namespace variable notation, even though most users aren't aware of what underlies an environment-variable references such as $env:HOME.
On occasion you see it used with a filesystem drive - e.g., ${c:\foo\file.txt} - but the fact that you can only use literal paths and that you cannot control the character encoding limits its usefulness.
It allows interesting uses, however; e.g.:
PS> $alias:foreach # Get the definition of alias 'foreach'
ForEach-Object
PS> $function:prompt # Get the body of the 'prompt' function
"PS $($executionContext.SessionState.Path.CurrentLocation)$('>' * ($nestedPromptLevel + 1)) ";
# .Link
# https://go.microsoft.com/fwlink/?LinkID=225750
# .ExternalHelp System.Management.Automation.dll-help.xml
# Define a function foo that echoes 'hi' and invoke it.
PS> $function:foo = { 'hi' }; foo
hi
Note:
Because ${<drive>:<path>} and ${<drive>:<path>} = <value> are equivalent to Get-Content -Path <drive>:<path> and Set-Content -Path <drive>:<path> <value>, paths are interpreted as wildcard expressions (because that's what -Path does, as opposed to -LiteralPath), which can cause problems with paths that look like wildcards - see this answer for an example and a workaround.
[1] Previously, the feature wasn't documented at all; GitHub docs issue #3343 led to the current documentation, albeit not in the way that said issue proposed.
$env is the Windows environment variables, the same as what you get when you do SET in a command prompt. There are a few that are PS-specific.
The variable is providing access to the Environment Provider. https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables?view=powershell-6
There are a bunch of other Providers that are described here: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_providers?view=powershell-6
As it says in the doco:
The model for data presentation is a file system drive. To use data
that the provider exposes, you view it, move through it, and change it
as though it were data on a hard drive. Therefore, the most important
information about a provider is the name of the drive that it
supports.
I was used to a few command line tricks in Windows that increased my productivity a lot.
Now I am told that I should move to PowerShell because it's more POWERful. It took me a while to get a little bit hang of it (objects, piping, etc.), and there are a lot of great tutorials on how to get a few things done.
However, some (relatively) basic trick still puzzle me. For instance, what is the equivalent of the FOR structure in PowerShell?
For example,
FOR %i IN (*.jpg) DO Convert %i -resize 800x300 resized/%i
The above line takes all of photos in a folder and uses the ImageMagick's Convert tool to resize the images and restores the resized imaged in a sub-folder called RESIZED.
In PowerShell I tried the command:
Dir ./ | foreach {convert $_.name -resize 800x300 resized/$_name}
This can't work despite all of the googling around I did. What is missing?
Note that / rather than \ is used as the path separator in this answer, which works on Windows too and makes the code compatible with the cross-platform PowerShell Core editions.
tl;dr:
$convertExe = './convert' # adjust path as necessary
Get-ChildItem -File -Filter *.jpg | ForEach-Object {
& $convertExe $_.Name -resize 800x300 resized/$($_.Name)
}
Read on for an explanation and background information.
The equivalent of:
FOR %i IN (*.jpg)
is:
Get-ChildItem -File -Filter *.jpg
or, with PowerShell's own wildcard expressions (slower, but more powerful):
Get-ChildItem -File -Path *.jpg # specifying parameter name -Path is optional
If you're not worried about restricting matches to files (as opposed to directories), Get-Item *.jpg will do too.
While dir works as a built-in alias for Get-ChildItem, I recommend getting used to PowerShell's own aliases, which follow a consistent naming convention; e.g., PowerShell's own alias for Get-ChildItem is gci
Also, in scripts it is better to always use the full command names - both for readability and robustness.
As you've discovered, to process the matching files in a loop you must pipe (|) the Get-ChildItem command's output to the ForEach-Object cmdlet, to which you pass a script block ({ ... }) that is executed for each input object, and in which $_ refers to the input object at hand.
(foreach is a built-in alias for ForEach-Object, but note that there's also a foreach statement, which works differently, and it's important not to confuse the two.)
There are 2 pitfalls for someone coming from the world of cmd.exe (batch files):
In PowerShell, referring to an executable by filename only (e.g., convert) does not execute an executable by that name located in the current directory, for security reasons.
Only executables in the PATH can be executed by filename only, and unless you've specifically placed ImageMagick's convert.exe in a directory that comes before the SYSTEM32 directory in the PATH, the standard Windows convert.exe utility (whose purpose is to convert FAT disk volumes to NTFS) will be invoked.
Use Get-Command convert to see what will actually execute when you submit convert; $env:PATH shows the current value of the PATH environment variable (equivalent of echo %PATH%).
If your custom convert.exe is indeed in the current directory, invoke it as ./convert - i.e., you must explicitly reference its location.
Otherwise (your convert.exe is either not in the PATH at all or is shadowed by a different utility) specify the path to the executable as needed, but note that if you reference that path in a variable or use a string that is single- or double-quoted (which is necessary if the path contains spaces, for instance), you must invoke with &, the call operator; e.g.,
& $convertExe ... or & "$HOME/ImageMagic 2/convert" ...
PowerShell sends objects through the pipeline, not strings (this innovation is at the heart of PowerShell's power). When you reference and object's property or an element by index as part of a larger string, you must enclose the expression in $(...), the subexpression operator:
resized/$($_.Name) - Correct: property reference enclosed in $(...)
resized/$_.Name - !! INCORRECT - $_ is stringified on its own, followed by literal .Name
However, note that a stand-alone property/index reference or even method call does not need $(...); e.g., $_.Name by itself, as used in the command in the question, does work, and retains its original type (is not stringified).
Note that a variable without property / index access - such as $_ by itself - does not need $(...), but in the case at hand $_ would expand to the full path. For the most part, unquoted tokens that include variable references are treated like implicitly double-quoted strings, whose interpolation rules are summarized in this answer of mine; however, many additional factors come into play, which are summarized here; edge cases are highlighted in this question.
At the end of the day, the safest choice is to double-quote strings that contain variable references or subexpressions:
"resized/$($_.Name)" - SAFEST
Use:
Get-ChildItem | foreach {convert $_.name -resize 800x300 resized/$($_.name)}
Or, perhaps, you need to pass the full name (with path), also showing a shorter syntax (using aliases):
gci | % {convert $_.fullname -resize 800x300 resized/$($_.name)}
Also, you might want to supply the full path to the executable.
Revised based on comments given below
There are many applications with the name "Convert". If I do
Get-Command Convert
on my computer. It shows me an app that is part of the Windows system. If PowerShell is running the wrong app on you, it's never going to work.
The solution will be to point PowerShell at the convert tool inside the ImageMagick program folder. A Google search on "ImageMagick PowerShell" will lead you to lots of people who have faced the same problem as you.
Let's say I wrote a PowerShell script that includes this commmand:
Get-ChildItem -Recurse
But instead I wrote:
Get-ChildItem -Re
To save time. After some time passed and I upgraded my PowerShell version, Microsoft decided to add a parameter to Get-ChildItem called "-Return", that for example returns True or False depending if any items are found or not.
In that virtual scenario, do I have I to edit all my former scripts to ensure that the script will function as expected? I understand Microsoft's attempt to save my typing time, but this is my concern and therefore I will probably always try to write the complete parameter name.
Unless of course you know something I don't. Thank you for your insight!
This sounds more like a rant than a question, but to answer:
In that virtual scenario, do I have I to edit all my former scripts to ensure that the script will function as expected?
Yes!
You should always use the full parameter names in scripts (or any other snippet of reusable code).
Automatic resolution of partial parameter names, aliases and other shortcuts are great for convenience when using PowerShell interactively. It lets us fire up powershell.exe and do:
ls -re *.ps1|% FullName
when we want to find the path to all scripts in the profile. Great for exploration!
But if I were to incorporate that functionality into a script I would do:
Get-ChildItem -Path $Home -Filter *.ps1 -Recurse |Select-Object -ExpandProperty FullName
not just for the reasons you mentioned, but also for consistency and readability - if a colleague of mine comes along and maybe isn't familiar with the shortcuts I'm using, he'll still be able to discern the meaning and expected output from the pipeline.
Note: There are currently three open issues on GitHub to add warning rules for this in PSScriptAnalyzer - I'm sure the project maintainers would love a hand with this :-)
I'm trying to export all CA certificates to a directory in Base64 format, I'm new to powershell, since I'm used to doing scripts with bash. Somehow I'm not seeing something that feels like it should be obvious.
This is my line so far,
#(Get-ChildItem -path Cert:\Localmachine\ca).ForEach({Export-Certificate -Type CERT -FilePath "C:\ssl\certs.d\$_.Thumbprint" -Cert "Cert:\LocalMachine\ca\$_.Thumbprint"})
I appreciate any help, as I'm trying to learn how to be idiomatic in PS4.
This line of code contains 3 issues:
First. String interpolation with object property. PS parser doesn't understand "$var.Property", it only understands $expression within "string". But since it's expression, and not just variable name, you can make PS evaluate your line with "$(something to evaluate)". In other words, your -FilePath should be:
-FilePath "C:\ssl\certs.d\$($_.Thumbprint)"
Second. Working with objects. PS underneath is full-blown .Net Framework. Even though many objects are represented in output in a simple, predefined way, actually they are |ed to output as complete live objects. According to MSDN, the -Cert parameter is a <Certificate>, not a string pointing to a certificate, so your -Cert should be simply
-Cert $_
Third. Arrays. Get-ChildItem underneath is nothing more than DirectoryInfo.GetFileSystemInfos() which returns an array of objects. So ideally, you don't need to wrap it with anything and it's possible to simply pipe it further (Get-ChildItem | Foreach-Object{...}). But many people have different tastes on PS syntax, so the form of (gci).ForEach({...}) (without #) has the right to live as well. But what you are doing in form of #(...) is creating a new array of one item being the array returned to you by gci. So technically, it just shouldn't work. It will though, because PS saves you from such mistakes automatically: in PS you can work with array of 1 item in the same way as with this item directly (unless explicitly specified opposite). To illustrate,
#(4).Length # returns 1
#(#(2,3)).Length # returns 2
#(,#(2,3)).Length #returns 1
Thus, your current syntax for Get-ChildItem is error-prone and relies on automatic PS error-handling sugar. I recommend to either remove # in the beginning, or to rewrite in form of
Get-ChildItem -...... | Foreach-Object {...}
Is it possible to get PowerShell to always output / instead of \? For example, I'd like the output of get-location to be C:/Documents and Settings/Administrator.
Update
Thanks for the examples of using replace, but I was hoping for this to happen globally (e.g. tab completion, etc.). Based on Matt's observation that the separator is defined by System.IO.Path.DirectorySeparatorChar which appears in practice and from the documentation to be read-only, I'm guessing this isn't possible.
It's a good question. The underlying .NET framework surfaces this as System.IO.Path.DirectorySeparatorChar, and it's a read/write property, so I figured you could do this:
[IO.Path]::DirectorySeparatorChar = '/'
... and that appears to succeed, except if you then type this:
[IO.Path]::DirectorySeparatorChar
... it tells you that it's still '\'. It's like it's not "taking hold". Heck, I'm not even sure that PowerShell honours that particular value even if it was changing.
I thought I'd post this (at the risk of it not actually answering your question) in case it helps someone else find the real answer. I'm sure it would be something to do with that DirectorySeparatorChar field.
Replace "\" with "/".
PS C:\Users\dance2die> $path = "C:\Documents and Settings\Administrator"
PS C:\Users\dance2die> $path.Replace("\", "/")
C:/Documents and Settings/Administrator
You could create a filter (or function) that you can pipe your paths to:
PS C:\> filter replace-slash {$_ -replace "\\", "/"}
PS C:\> Get-Location | replace-slash
C:/