I'm developing a cmdlet called Merge-Xsd that can merge similar XML schemas. It takes a list of paths, loads the schemas, merges them, and produces an XMLDocument as output.
All schemas of a particular file name are considered "similar", and so what I'm doing is getting all of the child items in a particular directory structure, grouping them according to the file name, and then trying to pass them to my custom cmdlet.
Grouping them is easy:
$grouping = Get-ChildItem -Recurse -Filter *.xsd |
Group-Object -Property Name -AsHashTable -AsString
However, processing them as part of the same pipeline is not. I've gotten as close as this:
$grouping.Keys |
ForEach-Object { ($grouping[$_] |
Select-Object -ExpandProperty FullName | Merge-Xsd).Save("C:\Out\$_") }
But what I'd really like to be able to do is use ForEach-Object directly after Group-Object to iterate over each group item, thus eliminating the need for the separate $grouping variable.
How can I use ForEach-Object to get the key/value pair while keeping each invocation of Merge-Xsd scoped to that particular key/value pair?
20150224 UPDATE:
The Merge-Xsd option set is extremely basic:
NAME
Merge-Xsd
SYNTAX
Merge-Xsd [-Path] <string[]> [<CommonParameters>]
It is really just intended for throwing a bunch of files at it in one go and having them merged into a single output, which is an XmlDocument. (I modeled the output off of ConvertTo-Xml.)
I think you could just nest it like this:
Get-ChildItem -Recurse -Filter *.xsd |
Group-Object -Property Name |
ForEach-Object {
($_.Group.FullName | Merge-Xsd).Save("C:\Out\$($_.Name)")
}
I don't have your cmdlet or files but in my limited testing this would work.
Some Explanation
I took out the -AsHash and -AsString parameters so we could deal directly with the group objects returned by Group-Object.
The $_.Group.FullName is more complex than it seems on first glance. $_ here refers to a single group object, since we're in a ForEach-Object. The group object contains a property called Name which is the name of the group, and a property called Group which is actually a collection of the the individual items within the group, so $_.Group is a collection.
From here, it would make sense to pipe that to ForEach-Object again, since each of the items in that collection will be a FileInfo object, and you want to get the FullName property to pass to Merge-Xsd.
Here we take advantage of some powershell magic. When you refer to $c.Property where $c is a collection of objects with a Property property, you get back a collection that consists of the property objects.
So $props = $c.Property is the same as:
$props = $c | ForEach-Object { $_.Property }
Knowing that, we can pipe $_.Group.FullName directly into Merge-Xsd to pass along all of the fullnames from all of the files in the group.
In that context, $_.Name still refers to the group object, so it's the name of the group, not the name of the file.
Related
I have this working, but need LastWriteTime and can't get it.
Get-ChildItem -Recurse | Select-String -Pattern "CYCLE" | Select-Object Path, Line, LastWriteTime
I get an empty column and zero Date-Time data
Select-String's output objects, which are of type Microsoft.PowerShell.Commands.MatchInfo, only contain the input file path (string), no other metadata such as LastWriteTime.
To obtain it, use a calculated property, combined with the common -PipelineVariable parameter,
which allows you to reference the input file at hand in the calculated property's expression script block as a System.IO.FileInfo instance as output by Get-ChildItem, whose .LastWriteTime property value you can return:
Get-ChildItem -File -Recurse -PipelineVariable file |
Select-String -Pattern "CYCLE" |
Select-Object Path,
Line,
#{
Name='LastWriteTime';
Expression={ $file.LastWriteTime }
}
Note how the pipeline variable, $file, must be passed without the leading $ (i.e. as file) as the -PipelineVariable argument . -PipelineVariable can be abbreviated to -pv.
LastWriteTime is a property of System.IO.FileSystemInfo, which is the base type of the items Get-ChildItem returns for the Filesystem provider (which is System.IO.FileInfo for files). Path and Line are properties of Microsoft.PowerShell.Commands.MatchInfo, which contains information about the match, not the file you passed in. Select-Object operates on the information piped into it, which comes from the previous expression in the pipeline, your Select-String in this case.
You can't do this as a (well-written) one-liner if you want the file name, line match, and the last write time of the actual file to be returned. I recommend using an intermediary PSCustomObject for this and we can loop over the found files and matches individually:
# Use -File to only get file objects
$foundMatchesInFiles = Get-ChildItem -Recurse -File | ForEach-Object {
# Assign $PSItem/$_ to $file since we will need it in the second loop
$file = $_
# Run Select-String on each found file
$file | Select-String -Pattern CYCLE | ForEach-Object {
[PSCustomObject]#{
Path = $_.Path
Line = $_.Line
FileLastWriteTime = $file.LastWriteTime
}
}
}
Note: I used a slightly altered name of FileLastWriteTime to exemplify that this comes from the returned file and not the match provided by Select-String, but you could use LastWriteTime if you wish to retain the original property name.
Now $foundMatchesInFiles will be a collection of files which have CYCLE occurring within them, the path of the file itself (as returned by Select-String), and the last write time of the file itself as was returned by the initial Get-ChildItem.
Additional considerations
You could also use Select-Object and computed properties but IMO the above is a more concise approach when merging properties from unrelated objects together. While not a poor approach, Select-Object outputs data with a type containing the original object type name (e.g. Selected.Microsoft.PowerShell.Commands.MatchInfo). The code may work fine but can cause some confusion when others who may consume this object in the future inspect the output members. LastWriteTime, for example, belongs to FileSystemInfo, not MatchInfo. Another developer may not understand where the property came from at first if it has the MatchInfo type referenced. It is generally a better design to create a new object with the merged properties.
That said this is a minor issue which largely comes down to stylistic preference and whether this object might be consumed by others aside from you. I write modules and scripts that many other teams in my organization consume so this is a concern for me. It may not be for you. #mklement0's answer is an excellent example of how to use computed properties with Select-Object to achieve the same functional result as this answer.
I am trying to utilize PowerShell to audit all of our security group members in AD. I have been trying to get Get-ADGroupMember to work but anytime I try it, it returns the message 'Cannot find an object with identity 'groupName' under: 'DC=xxxx,DC=xxxx,DC=xxxx,DC=xxxx'.
Ive tried the following with no luck:
$groupNames = 'groupName1' , 'groupName2' , 'groupName3'
foreach ($group in $groupNames) {
Get-AdGroupMember -Identity $group
}
Has anyone successfully compiled a list of group members in security groups from AD and exported them into a .CSV?
There are few things to consider when querying AD groups using the Get-AdGroup* commands.
The -Identity parameter only accepts values that match an object's Guid, ObjectSid, DistinguishedName, or SamAccountName. If your input is something other than one of those attribute values, you will either need to run another command to retrieve the proper data or change your list.
-Identity only accepts a single value, which means if you want to supply a list of values, you need to loop through them.
Get-AdGroupMember does not output as many attribute/value pairs as Get-AdUser. You cannot force it to output more attributes than it does. It does not have a -Properties parameter like Get-AdUser. Sometimes it requires using both commands to get all of the required data.
You can send Get-Ad* output to CSV using Export-Csv. If you do not use any property filtering like with Select-Object, the returned property names will be the columns of the CSV. The associated values of the properties will appear in rows with each row representing one returned object. You can choose to either send the entire results of the command once to the CSV or each time the command runs using Export-Csv -Append.
Use Select-Object to only output properties you care about. Select-Object Property outputs a custom object that includes only the property Property and the value(s) of Property for each object returned. If you only want to return the value rather than a custom object, you can use Select-Object -Expand Property.
Get-Content can be used to read a file. If the file contains only a list of values, perhaps SamAccountName values, you can use Get-Content file.txt to retrieve that list. The list will be an array that can be looped through.
Since Get-AdUser can be verbose, it is wise to use the -Properties parameter to explicitly list any extra properties beyond the default set you want to return. -Properties * will return all properties, but that is not best practice.
Given the above considerations, I would do the following:
$groupNames = 'groupName1' , 'groupName2' , 'groupName3'
# Alternatively, if you have a file (file.txt) with your group names listed as one group per line
$groupNames = Get-Content file.txt
# The Foreach-Object section is only needed if $groupNames does not contain a valid -Identity value
# The Filter below uses Name attribute as an example because it assumes $groupNames contains Name attribute values. If it contains another attribute, update the filter accordingly.
$SamAccountNames = $groupNames | Foreach-Object {
Get-AdGroup -Filter "Name -eq '$_'" | Select-Object -Expand SamAccountName
}
# Storing the loop output into a variable is efficient provided you have enough memory for the operation.
# Alternatively, you can just pipe the `Get-AdGroupMember` into `Export-Csv -Append` but that could be a lot of writes!
$output = foreach ($group in $SamAccountNames) {
Get-AdGroupMember -Identity $group # You can use Select-Object here for specific properties
}
$output | Export-Csv -Path output.csv -NoTypeInformation
I need to script up some things in PowerCLI (VMWare's bolt on to PowerShell). Basically we have a server cluster with three hosts. Each host has multiple virtual switches. Each virtual switch has multiple vlans ('port groups' in VMWare speak). I need to audit the fact that the same port groups exist on each host (so things keep working if the VM is moved).
Step 1 to achieving this is would be to know that the port group name exists on each of the three host machines.
I'm falling over with how to filter some objects out of all the ones returned by a cmdlet, based on number of results returned from a property of those objects. I then need to perform further operations with original object type that passes the filter test to go on down the pipeline.
To give some specifics, this an example showing 'Some PortGroup Name' and the three hosts it exists on (and as a bonus, the vSwitch):
Get-VirtualPortGroup -Name 'Some PortGroup Name' |
Select-Object Name, VMHostID, VirtualSwitchId
produces the output
Name VMHostId VirtualSwitchId
---- -------- ---------------
Some PortGroup Name HostSystem-host-29459 key-vim.host.VirtualSwitch-vSwitch6
Some PortGroup Name HostSystem-host-29463 key-vim.host.VirtualSwitch-vSwitch6
Some PortGroup Name HostSystem-host-29471 key-vim.host.VirtualSwitch-vSwitch6
Instead of 3, I'm starting with the 1849 port group names that are being returned by Get-VirtualPortGroup. I need a pipeline to whittle the number of VirtualPortGroup objects down to a collection consisting of only those objects where a count of the 'VMHostId' property is less than 3, and pass the remaining VirtualPortGroup objects down the pipeline for further processing.
This seems simple enough to do. I'm still failing though.
The following almost works. Piping it to measure shows a count of 229, instead of the original 1849 (so it's definitely filtered a lot out, and is possibly correctly returning the subset I'm after...?). The problem is, the object type is now a 'Group' or something at this point in the pipeline, and doesn't have all the properties and methods of the original Get-VirtualPortGroup objects.
Get-VirtualPortGroup |
Group-Object -Property Name |
Where-Object $_.Count -lt 3
Bolting a | Select-Object -ExpandProperty Group to the end of the above seemed promising, except it then seems to return the entire collection of Get-VirtualPortGroup objects as though I had done no filtering in there at all....
Am I doing something fundamentally wrong?
How can I filter out objects based on the count of the number of results returned by a specific property of an object, but still pass the original object type down the pipe?
Your approach is correct, but you got the Where-Object syntax wrong. The abbreviated syntax is:
Where-Object <property> <op> <value>
without the current object variable ($_). In your case that would be:
Where-Object Count -lt 3
Otherwise you must use scriptblock notation:
Where-Object { $_.Count -lt 3 }
This should do what you want:
Get-VirtualPortGroup |
Group-Object -Property Name |
Where-Object { $_.Count -lt 3 } |
Select-Object -Expand Group
I have the power shell below that selectes certain fields
dir -Path E:\scripts\br\test | Get-FileMetaData | Select-Object name, Comments, Path, Rating
what i want to do is utilize Name,Comments,Path,Rating in further Pipes $_.name etc dosnt work
If I understand your question correctly, you want to do something with the output of Select-Object, but you want to do it in a pipeline.
To do this, you need to pass the output down the pipeline into a Cmdlet that accepts pipeline input (such as ForEach-Object). If the next operation in the pipeline does not accept pipeline input, you will have to set the output to a variable and access the information through the variable,
Using ForEach-Object
In this method, you will be processing each object individually. This will be similar to the first option in Method 1 (that is, dealing with individual items in the collection of items returned by Select-Object).
dir | Get-FileMetaData | Select-Object Name,Comments,Path,Rating | ForEach-Object {
# Do stuff with $_
# Note that $_ is a single item in the collection returned by Select-Object
}
The variable method is included in case your next Cmdlet does not accept pipeline input.
Using Variable
In this method, you will treat $tempVariable as an array and you can operate on each item. If need be, you can actually access each column individually, getting everything at once.
$tempVariable = dir | Get-FileMetaData | Select-Object Name,Comments,Path,Rating
# Do stuff with each Name by using $tempVariable[i].Name, etc.
# Or do stuff with all Names by using $tempVariable.Name, etc.
The following code gives me an array of PSCustomObjects, how can I get it to return an array of Strings?
$files = Get-ChildItem $directory -Recurse | Select-Object FullName | Where-Object {!($_.psiscontainer)}
(As a secondary question, what's the psiscontainer part for? I copied that from an example online)
Post-Accept Edit: Two great answers, wish I could mark both of them. Have awarded the original answer.
You just need to pick out the property you want from the objects. FullName in this case.
$files = Get-ChildItem $directory -Recurse | Select-Object FullName | Where-Object {!($_.psiscontainer)} | foreach {$_.FullName}
Edit: Explanation for Mark, who asks, "What does the foreach do? What is that enumerating over?"
Sung Meister's explanation is very good, but I'll add a walkthrough here because it could be helpful.
The key concept is the pipeline. Picture a series of pingpong balls rolling down a narrow tube one after the other. These are the objects in the pipeline. Each stage of pipeline--the code segments separated by pipe (|) characters--has a pipe going into it and pipe going out of it. The output of one stage is connected to the input of the next stage. Each stage takes the objects as they arrive, does things to them, and sends them back out into the output pipeline or sends out new, replacement objects.
Get-ChildItem $directory -Recurse
Get-ChildItem walks through the filesystem creating FileSystemInfo objects that represent each file and directory it encounters, and puts them into the pipeline.
Select-Object FullName
Select-Object takes each FileSystemInfo object as it arrives, grabs the FullName property from it (which is a path in this case), puts that property into a brand new custom object it has created, and puts that custom object out into the pipeline.
Where-Object {!($_.psiscontainer)}
This is a filter. It takes each object, examines it, and sends it back out or discards it depending on some condition. Your code here has a bug, by the way. The custom objects that arrive here don't have a psiscontainer property. This stage doesn't actually do anything. Sung Meister's code is better.
foreach {$_.FullName}
Foreach, whose long name is ForEach-Object, grabs each object as it arrives, and here, grabs the FullName property, a string, from it. Now, here is the subtle part: Any value that isn't consumed, that is, isn't captured by a variable or suppressed in some way, is put into the output pipeline. As an experiment, try replacing that stage with this:
foreach {'hello'; $_.FullName; 1; 2; 3}
Actually try it out and examine the output. There are four values in that code block. None of them are consumed. Notice that they all appear in the output. Now try this:
foreach {'hello'; $_.FullName; $ x = 1; 2; 3}
Notice that one of the values is being captured by a variable. It doesn't appear in the output pipeline.
To get the string for the file name you can use
$files = Get-ChildItem $directory -Recurse | Where-Object {!($_.psiscontainer)} | Select-Object -ExpandProperty FullName
The -ExpandProperty parameter allows you to get back an object based on the type of the property specified.
Further testing shows that this did not work with V1, but that functionality is fixed as of the V2 CTP3.
For Question #1
I have removed "select-object" portion - it's redundant and moved "where" filter before "foreach" unlike dangph's answer - Filter as soon as possible so that you are dealing with only a subset of what you have to deal with in the next pipe line.
$files = Get-ChildItem $directory -Recurse | Where-Object {!$_.PsIsContainer} | foreach {$_.FullName}
That code snippet essentially reads
Get all files full path of all files recursively (Get-ChildItem $directory -Recurse)
Filter out directories (Where-Object {!$_.PsIsContainer})
Return full file name only (foreach {$_.FullName})
Save all file names into $files
Note that for foreach {$_.FullName}, in powershell, last statement in a script block ({...}) is returned, in this case $_.FullName of type string
If you really need to get a raw object, you don't need to do anything after getting rid of "select-object". If you were to use Select-Object but want to access raw object, use "PsBase", which is a totally different question(topic) - Refer to "What's up with PSBASE, PSEXTENDED, PSADAPTED, and PSOBJECT?" for more information on that subject
For Question #2
And also filtering by !$_.PsIsContainer means that you are excluding a container level objects - In your case, you are doing Get-ChildItem on a FileSystem provider(you can see PowerShell providers through Get-PsProvider), so the container is a DirectoryInfo(folder)
PsIsContainer means different things under different PowerShell providers;
e.g.) For Registry provider, PsIsContainer is of type Microsoft.Win32.RegistryKey
Try this:
>pushd HKLM:\SOFTWARE
>ls | gm
[UPDATE] to following question: What does the foreach do? What is that enumerating over?
To clarify, "foreach" is an alias for "Foreach-Object"
You can find out through,
get-help foreach
-- or --
get-alias foreach
Now in my answer, "foreach" is enumerating each object instance of type FileInfo returned from previous pipe (which has filtered directories). FileInfo has a property called FullName and that is what "foreach" is enumerating over.
And you reference object passed through pipeline through a special pipeline variable called "$_" which is of type FileInfo within the script block context of "foreach".
For V1, add the following filter to your profile:
filter Get-PropertyValue([string]$name) { $_.$name }
Then you can do this:
gci . -r | ?{!$_.psiscontainer} | Get-PropertyName fullname
BTW, if you are using the PowerShell Community Extensions you already have this.
Regarding the ability to use Select-Object -Expand in V2, it is a cute trick but not obvious and really isn't what Select-Object nor -Expand was meant for. -Expand is all about flattening like LINQ's SelectMany and Select-Object is about projection of multiple properties onto a custom object.