I have a problem in my script:
here I set a var remotefilehash:
$remotefilehash = $remotefiles -replace '^[a-f0-9]{32}( )', '$0= ' | ConvertFrom-StringData
this creates a hashtable
I then compare those hash's to what I have locally in a hashset
# Create a hash set from the local hashes.
$localHashSet = [System.Collections.Generic.HashSet[string]] $localmd5
# Loop over all remote hashes to find those not among the local hashes.
$diffmd5 = $remotefilehash.Keys.Where({ -not $localHashSet.Contains($_) })
this gets me the hashkeys but I subsequently also need to get the hash values for those keys that are found in the above where ...
this creates a hashtable
Actually, it creates an array of hashtables, because ConvertFrom-StringData turns each pipeline input object into a separate hashtable.
To create a single hashtable, join the input lines to form a single string first - note how -join "`n" is applied to the result of the -replace operation in order to form a single, multi-line string:
$remotefilehash =
($remotefiles -replace '^[a-f0-9]{32}( )', '$0= ' -join "`n") |
ConvertFrom-StringData
To enumerate the key-value pairs instead of just the keys (.Keys) of a hashtable(-like) object, you need to use its .GetEnumerator() method:
$diffmd5 =
$remotefilehash.GetEnumerator().Where({ -not $localHashSet.Contains($_.Key) })
The reason that an enumerator must be requested explicitly is that PowerShell by default considers a hashtable/dictionary a single object that is passed as a whole through the pipeline / passed to enumeration methods such as the .Where() and .ForEach() array methods.
Note that the output of the above command is not itself a hashtable, but a regular, array-like collection (of type [System.Collections.ObjectModel.Collection[psobject]) whose elements happen to be key-value pair objects.
As such, this output is enumerated in the pipeline.
Related
I have an array with data separated by a comma. I need to transpose it so the first part before the comma for each line is joined together by a delimiter as one line and the same for a second part. Example:
AC-2.22,CCI-000012
AC-5.1,CCI-000036
AC-5.3,CCI-001380
I want to have 2 separate variables like so:
variable 1 = AC-2.22; AC-5.1; AC-5.3
Variable 2 = CCI-000012; CCI-000036; CCI-001380
I know this should be simple but I've been staring at code all day and I just want to go eat dinner and goto sleep.
Thanks in advance
Based on the $array of the helpful answer from Santiago Squarzon, you might also use the ConvertFrom-Csv cmdlet to transpose the data using member-access enumeration:
$data = ConvertFrom-Csv $Array -Header var1, var2
$data.var1 -Join ';'
AC-2.22;AC-5.1;AC-5.3
$data.var2 -Join ';'
CCI-000012;CCI-000036;CCI-001380
This is not too hard using .Where method with Split mode:
$array = #(
'AC-2.22,CCI-000012'
'AC-5.1,CCI-000036'
'AC-5.3,CCI-001380'
)
$i = $false
$var1, $var2 = $array.Split(',').Where({ ($i = -not $i) }, 'Split')
$var1 -join '; '
$var2 -join '; '
.Split method works in this case thanks to Member-Access Enumeration, because arrays don't have an .Split method the same is called over each element of the array:
When you use the member-access operator on any object and the specified member exists on that object, the member is invoked. For property members, the operator returns the value of that property. For method members, the operator calls that method on the object.
When you use the member-access operator on a list collection object that doesn't have the specified member, PowerShell automatically enumerates the items in that collection and uses the member-access operator on each enumerated item.
A Microsoft utility returns strings in the following format:
"Author: First.Last; Name: RootConfiguration; Version: 2.0.0; GenerationDate: 06/01/2022 13:18:10; GenerationHost: Server;"
I would like to convert those strings into simple objects. If this were true JSON, I'd just use ConvertFrom-JSON. To reinvent the wheel as little as possible, what's the most straightforward way to convert that into an object (with keys Author, Name, Version, GenerationDate, GenerationHost, with the obvious values. It's fine if the values are all treated as dumb strings.
If "you just have to grind it out by tokenizing the string bite by bite" is the answer, I can do that, but it seems there should be a simpler way, like if I could tell ConvertFrom-JSON (or even ConvertFrom-String!) "Do your thing, but process the semicolons as newlines, accept spaces on the right hand side, etc."
A solution that combines manual parsing with ConvertFrom-StringData, but note that input order of the entries isn't preserved, given that the latter returns a [hashtable] instance with inherently unordered entries:
# Sampe input string.
$str = 'Author: First.Last; Name: RootConfiguration; Version: 2.0.0; GenerationDate: 06/01/2022 13:18:10; GenerationHost: Server;'
# Replace ":" with "=", split into individual lines, so
# that ConvertFrom-StringData recognizes the format.
$str -replace ': ', '=' -replace '; ?', "`n" | ConvertFrom-StringData
# Note: The above outputs a [hashtable].
# You could cast it to [pscustomobject], as shown below,
# but the input order of entries is lost either way.
As zett42 points out, if the values (as opposed to the keys) in the input string contained \ chars., they'd need to be doubled in order to be retained as such - see his comment below.
A solution with manual parsing only:
# Sampe input string.
$str = 'Author: First.Last; Name: RootConfiguration; Version: 2.0.0; GenerationDate: 06/01/2022 13:18:10; GenerationHost: Server;'
# Initialize an ordered hashtable (dictionary)
$dict = [ordered] #{}
# Split the string by ";", then each entry into key and value by ":".
$str -split '; ?' |
ForEach-Object { $key, $value = $_ -split ': ', 2; $dict[$key] = $value }
# Convert the ordered hashtable (dictionary) to a custom object.
[pscustomobject] $dict
I usually don't answer questions that don't have a coding attempt but, figured this might help others. Given that the delimiter is a semicolon, I was thinking of converting to CSV first but, would have to worry about the header next. So, instead of converting to CSV, we can use the delimiter to split the results at that and process the values one at a time:
"Author: First.Last; Name: RootConfiguration; Version: 2.0.0; GenerationDate: 06/01/2022 13:18:10; GenerationHost: Server;".Split(";").Trim() |
ForEach-Object -Process {
$header,$value = $_ -split ":",2
New-Object -TypeName PSCustomObject #{
$header = $value
}
} | ConvertTo-Json
To make this work we need to split at just the first colon (:) leaving the rest intact; incase there's others in the value like you see in the GenerationDate property.
This was achieved using $_ -split ":",2.
Finally, the rest was just left to assign the header to and value to a PSCustomObject and conver the results to JSON using ConvertTo-Json.
Note: I am restricted to a "strict language mode" on my work system so it's best to use the type-accelarator of [PSCustomObject]#{..} to create the object, rather than New-Object.
Complementing the existing helpful answers, here is another one using the Regex.Matches() function:
$testInput = 'Author: First.Last; Name: RootConfiguration; Version: 2.0.0; GenerationDate: 06/01/2022 13:18:10; GenerationHost: Server;'
# Create a temporary, ordered Hashtable to collect keys and values in the original order.
$ht = [ordered] #{}
# Use a regular expression to find all key/value pairs.
foreach( $match in [regex]::Matches( $testInput, '\s*([^:]+):\s*([^;]+);') ) {
# Enter a key/value pair into the Hashtable
$ht[ $match.Groups[1] ] = $match.Groups[2]
}
# Convert the temporary Hashtable to PSCustomObject.
[PSCustomObject] $ht
Output:
Author : First.Last
Name : RootConfiguration
Version : 2.0.0
GenerationDate : 06/01/2022 13:18:10
GenerationHost : Server
The RegEx pattern consists of two capturing groups ( ), where the first one captures a key and the second one captures a value.
For a detailed explanation see regex101, where you can also play around with the pattern.
I've seen the # symbol used in PowerShell to initialise arrays.
What exactly does the # symbol denote and where can I read more about it?
In PowerShell V2, # is also the Splat operator.
PS> # First use it to create a hashtable of parameters:
PS> $params = #{path = "c:\temp"; Recurse= $true}
PS> # Then use it to SPLAT the parameters - which is to say to expand a hash table
PS> # into a set of command line parameters.
PS> dir #params
PS> # That was the equivalent of:
PS> dir -Path c:\temp -Recurse:$true
PowerShell will actually treat any comma-separated list as an array:
"server1","server2"
So the # is optional in those cases. However, for associative arrays, the # is required:
#{"Key"="Value";"Key2"="Value2"}
Officially, # is the "array operator." You can read more about it in the documentation that installed along with PowerShell, or in a book like "Windows PowerShell: TFM," which I co-authored.
While the above responses provide most of the answer it is useful--even this late to the question--to provide the full answer, to wit:
Array sub-expression (see about_arrays)
Forces the value to be an array, even if a singleton or a null, e.g. $a = #(ps | where name -like 'foo')
Hash initializer (see about_hash_tables)
Initializes a hash table with key-value pairs, e.g.
$HashArguments = #{ Path = "test.txt"; Destination = "test2.txt"; WhatIf = $true }
Splatting (see about_splatting)
Let's you invoke a cmdlet with parameters from an array or a hash-table rather than the more customary individually enumerated parameters, e.g. using the hash table just above, Copy-Item #HashArguments
Here strings (see about_quoting_rules)
Let's you create strings with easily embedded quotes, typically used for multi-line strings, e.g.:
$data = #"
line one
line two
something "quoted" here
"#
Because this type of question (what does 'x' notation mean in PowerShell?) is so common here on StackOverflow as well as in many reader comments, I put together a lexicon of PowerShell punctuation, just published on Simple-Talk.com. Read all about # as well as % and # and $_ and ? and more at The Complete Guide to PowerShell Punctuation. Attached to the article is this wallchart that gives you everything on a single sheet:
You can also wrap the output of a cmdlet (or pipeline) in #() to ensure that what you get back is an array rather than a single item.
For instance, dir usually returns a list, but depending on the options, it might return a single object. If you are planning on iterating through the results with a foreach-object, you need to make sure you get a list back. Here's a contrived example:
$results = #( dir c:\autoexec.bat)
One more thing... an empty array (like to initialize a variable) is denoted #().
The Splatting Operator
To create an array, we create a variable and assign the array. Arrays are noted by the "#" symbol. Let's take the discussion above and use an array to connect to multiple remote computers:
$strComputers = #("Server1", "Server2", "Server3")<enter>
They are used for arrays and hashes.
PowerShell Tutorial 7: Accumulate, Recall, and Modify Data
Array Literals In PowerShell
I hope this helps to understand it a bit better.
You can store "values" within a key and return that value to do something.
In this case I have just provided #{a="";b="";c="";} and if not in the options i.e "keys" (a, b or c) then don't return a value
$array = #{
a = "test1";
b = "test2";
c = "test3"
}
foreach($elem in $array.GetEnumerator()){
if ($elem.key -eq "a"){
$key = $elem.key
$value = $elem.value
}
elseif ($elem.key -eq "b"){
$key = $elem.key
$value = $elem.value
}
elseif ($elem.key -eq "c"){
$key = $elem.key
$value = $elem.value
}
else{
Write-Host "No other value"
}
Write-Host "Key: " $key "Value: " $value
}
Some objects has been saved to a txt.file
looking like this:
#{flightNumber=01; flightDate=2010-01-10; flightIdentification=201001}
#{flightNumber=01; flightDate=2010-01-10; flightIdentification=201002}
and I'm trying to read them in another program and convert them back into objects. What bothers me is that it understands each of the "objects" as a string and I have been unable to cast it into an object.
$list = Get-Content -Path 'C:\Users\XXXXX\Downloads\TemplateObject.txt'
foreach (#object in $list) {
Write-Host $object.flightNumber
}
From what I've shown, I would expect to see 2 different objects with the variables flightNumber, flightDate and flightIdentification
I've tried piping it by using ConvertFrom-StringData
I've tried casting to an object
I expect 2 separate objects containing 3 variables in each.
Don't pipe objects directly to files!
As has been pointed out, take advantage of built-in options for serialization to disk, like ConvertTo-Csv/Export-Csv for flat objects, ConvertTo-Json or Export-Clixml for more complex objects.
As a one-off thing, if you need to recover and re-encode this data, you could use the regex -replace operator to add quotes around the values, at which point the parser should accept them as hashtable entries and you can cast it to an object:
$string = '#{flightNumber=01; flightDate=2010-01-10; flightIdentification=201001}'
# Place double-quotes around anything found between a `=` and `;` or `}`
$quotedString = $string -replace '(?<=\=)([^=;}]+)(?=\s*(?:;|}))', '"$1"'
# Parse the resulting string as if it was PowerShell code
$errors = #()
$objectAST = [System.Management.Automation.Language.Parser]::ParseInput($quotedString, [ref]$null,[ref]$errors)
$objects = if(-not $errors){
# This is pretty dangerous, you should NEVER do this in a production script
$objectAST.GetScriptBlock.Invoke() |ForEach-Object {
[pscustomobject]$_
}
}
# This variable now contains the re-animated objects
$objects
You can convert a string to a hashtable using convertfrom-stringdata after some manipulation:
$a = '#{flightNumber=01; flightDate=2010-01-10; flightIdentification=201001}'
$a = $a -replace '#{' -replace '}' -replace ';',"`n" | ConvertFrom-StringData
[pscustomobject]$a
flightNumber flightIdentification flightDate
------------ -------------------- ----------
01 201001 2010-01-10
I'm trying to export some lists from Sharepoint into a CSV file. My goal is to make a single function that is flexible enough to take a List name, identifier for the CSV file, and a list of FieldValues to export, and then produce a CSV file. Here's what I have so far:
function getTableData($_ctx, [string]$_colName)
{
$list = $_ctx.Web.Lists.GetByTitle("$_colName")
$camlQuery = [Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery(100)
$colItems = $list.GetItems($camlQuery)
$_ctx.Load($colItems)
$_ctx.ExecuteQuery();
return $colItems
}
# More will go into this array, but for now a single entry is sufficient for testing purposes
$mstLists = #("GroupMst", "Groups", #("Title", "GroupCode"))
$cols = #()
foreach($col in $mstLists[0][2])
{
$cols += #{Name=$col;expression={$_[$col];}}
}
$cols
# Grab all items from a list
getListData $ctx $mstLists[0][0] |
%{ select-object -input $_ -prop $cols } |
Export-Csv -Path ($export_path + '\' + $current_date + '_' + $mstLists[0][1] + '.csv') -Encoding UTF8 -NoTypeInformation
The problem I'm having is in the loop that populates $cols. Basically, each item needs to look like #{Name="Title";expression={$_["Title"];}} in order for select-object in the ForEach to grab the proper fields from the List. Unfortunately $cols ends up being looking like this:
Name Value
---- -----
expression $_[$col];
Name Title
expression $_[$col];
Name GroupCode
Which (somehow) produces a CSV file that looks like this:
"Title","GroupCode"
"LA","LA"
"NY","NY"
"TK","TK"
When the output needs to look like this:
"Title","GroupCode"
"Los Angeles","LA"
"New York","NY"
"Tokyo","TK"
I know the field names are correct - if I hardcode them in like so...
# Grab all items from a list
getListData $ctx $mstLists[0][0] |
%{ select-object -input $_ -prop `
#{Name="Title";expression={$_["Title"];}}, `
#{Name='GroupCode';expression={$_["GroupCode"];}}; } |
Export-Csv -Path ($export_path + '\' + $current_date + '_' + $mstLists[0][1] + '.csv') -Encoding UTF8 -NoTypeInformation
...then I get the desired CSV output. I just can't figure out how to get $_[$col] to instead return $_["Title"]
Not sure what the $mstLists[0][2] meant to refer to, but the following code seems to give what you are after...
$mstLists = #("GroupMst", "Groups", #("Title", "GroupCode"))
$cols = #()
foreach($col in $mstLists[2])
{
$cols += #{Name=$col; Expression = [scriptblock]::Create('$_["{0}"]' -f $col)}
}
$cols
which gives...
Name Value
---- -----
Name Title
Expression $_["Title"]
Name GroupCode
Expression $_["GroupCode"]
In your response to andyb in the comments, you say that each item of the array will follow the format
#("ListName", "CSVFileID", #("Field1", "Field2", "Etc..."))
and that $mstLists[0][2] "refers to the list of fields in the first item in the array."
The problem is that it doesn't refer to the list of fields in the first item of the array, because the first item of the array isn't a list of anything, it's the string GroupMst. When you index into a string, you get the character indicated by the index. Since $mstLists[0] is a string, $mstLists[0][2] returns the third character of that string, which is o.
I suppose you were expecting that the # operator would make the array in the parentheses a single item, which becomes the first element of $mstLists? It doesn't. All the # does is ensure that the expression in the parentheses is evaluated as an array rather than a scalar. So, with $a = ('string'), $a is a string, whereas with $a = #('string'), $a is an array with a single string element.
However, since ("GroupMst", "Groups", #("Title", "GroupCode")) evaluates to an array anyway, putting an # in front of it is redundant. Either way you're still assigning a literal array to the variable. $mstLists is an array of three elements:
Element 0 is the string GroupMst
Element 1 is the string Groups
Element 2 is an array of the strings Title and GroupCode
What you want to do is use , as a unary operator:
$mstLists = , ("GroupMst", "Groups", #("Title", "GroupCode"))
Now $mstLists is an array of a single item whose value is the array described in the bulleted list above, and $mstLists[0][2] evaluates to an array of the strings Title and GroupCode, as you were expecting.
Note that , works as a unary or binary operator that returns an array of the operands. To return a single-element array, you use it as a unary operator in front of that element. If you have multiple literal arrays that you want to assign to mstLists, you only need commas between them, not the one in front:
$mstLists = ("ListName", "CSVFileID", #("Field1", "Field2", "Etc...")), ("ListName2", "CSVFileID2", #("Field1", "Field2", "Etc..."))
That addresses the main problem. That still won't quite give you what you want, because $col won't get interpolated in the Expression scriptblock, so Expression will always be literally $_[$col]. However, in order to figure out how to do what you actually want to do, it would be helpful to see a sample of the contents of $ctx.