I have a script which I'm trying to modify, which adds objects to a firewall via powershell, using Invoke-RestMethod
The current script has the following code;
#Import CSV and set variables
$csv = import-csv C:\Powershell\groups.csv
# RESTful API Call
$csv | ForEach-Object {
$Name = $_.name
$Member1 =$_.member1
$Member2 =$_.member2
Invoke-RestMethod -Uri https://172.16.16.16:4444/webconsole/APIController?reqxml=<Request><Login><Username>admin</Username><Password>password</Password></Login><Set%20operation=%27add%27><IPHostGroup><Name>$Name</Name><IPFamily>IPv4</IPFamily><HostList><Host>$Member1</Host><Host>$Member2</Host></IPHostGroup></Set></Request>
}
I am wanting to import the hostgroups via groups.csv which (in my test) has 3 columns as follows;
Name,Member1,Member2
TestGroup,TestHost1,TestHost2
TestGroup2,TestHost3,TestHost4
etc.
My problem is that in the real data, there are varying amount of hosts in each group, some have hundreds. I'm not sure how to get these into the command without defining a variable for each possible member. Even then, say I created $Member(s) all the way to 200 (be gentle, I'm not a real coder!) and then imported them in manually one by one in the Invoke-Restmethod command (Might as well do it by hand at that point!) I'm not sure the command would handle the blank inputs in the cases where there were only a few hosts in the group.
(In other words if my csv had the following entries;)
Name,Member1,Member2,Member3,Member4
TestGroup,TestHost1,TestHost2,TestHost3,TestHost4
TestGroup2,TestHost5,TestHost6
TestGroup3,TestHost7
And I did;
# RESTful API Call
$csv | ForEach-Object {
$Name = $_.name
$Member1 =$_.member1
$Member2 =$_.member2
$Member3 =$_.member3
$Member4 =$_.member4
The Rest call for the third group would end up running as;
Invoke-RestMethod -Uri https://172.16.16.16:4444/webconsole/APIController?reqxml=<Request><Login><Username>admin</Username><Password>password</Password></Login><Set%20operation=%27add%27><IPHostGroup><Name>TestGroup3</Name><IPFamily>IPv4</IPFamily><HostList><Host>TestHost7</Host><Host></Host><Host></Host><Host></Host></IPHostGroup></Set></Request>
Can anyone point me in the direction of a better way of doing this?
You can get all member names using .PSObject.Properties.Name
Example:
$Csv = Import-Csv -Path 'C:\Powershell\groups.csv'
# Request XML template
$RequestTpl = #'
<Request>
<Login>
<Username>admin</Username>
<Password>password</Password>
</Login>
<Set%20operation=%27add%27>
<IPHostGroup>
<Name>{0}</Name>
<IPFamily>IPv4</IPFamily>
<HostList>
{1}
</HostList>
</IPHostGroup>
</Set>
</Request>
'#
# Host list XML template
$RequestHostListTpl = '<Host>{0}</Host>'
$Csv | ForEach-Object {
<#
Get names of all the properties in the current object
Leave only those that don't match '^Name$' regex.
-match, when operates on collections, returns matched items
You can use
$_.PSObject.Properties.Name | Where-Object {$_ -ne 'Name'}
but it's a bit slower.
#>
$Members = #($_.PSObject.Properties.Name) -notmatch '^Name$'
# Build list of hosts
$RequestHostList = foreach ($item in $Members) {
# Only add item if it's not empty
if ($_.$item) {
$RequestHostListTpl -f $_.$item
}
}
# Build request XML
$Request = $RequestTpl -f $_.Name, -join $RequestHostList
# Remove newlines to make it one long string
$Request = $Request -replace '\r|\n'
# Show resulting url
"Invoke-RestMethod -Uri https://172.16.16.16:4444/webconsole/APIController?reqxml=$Request"
# Uncomment to actually invoke API call
#Invoke-RestMethod -Uri "https://172.16.16.16:4444/webconsole/APIController?reqxml=$Request"
}
Related
I am looking for a way to randomize a specific string in a huge file by using predefined strings from array, without having to write temporary file on disk.
There is a file which contains the same string, e.g. "ABC123456789" at many places:
<Id>ABC123456789</Id><tag1>some data</tag1><Id>ABC123456789</Id><Id>ABC123456789</Id><tag2>some data</tag2><Id>ABC123456789</Id><tag1>some data</tag1><tag3>some data</tag3><Id>ABC123456789</Id><Id>ABC123456789</Id>
I am trying to randomize that "ABC123456789" string using array, or list of defined strings, e.g. "#('foo','bar','baz','foo-1','bar-1')". Each ABC123456789 should be replaced by randomly picked string from the array/list.
I have ended up with following solution, which is working "fine". But it definitely is not the right approach, as it do many savings on disk - one for each replaced string and therefore is very slow:
$inputFile = Get-Content 'c:\temp\randomize.xml' -raw
$checkString = Get-Content -Path 'c:\temp\randomize.xml' -Raw | Select-String -Pattern '<Id>ABC123456789'
[regex]$pattern = "<Id>ABC123456789"
while($checkString -ne $null) {
$pattern.replace($inputFile, "<Id>$(Get-Random -InputObject #('foo','bar','baz','foo-1','bar-1'))", 1) | Set-Content 'c:\temp\randomize.xml' -NoNewline
$inputFile = Get-Content 'c:\temp\randomize.xml' -raw
$checkString = Get-Content -Path 'c:\temp\randomize.xml' -Raw | Select-String -Pattern '<Id>ABC123456789'
}
Write-Host All finished
The output is randomized, e.g.:
<Id>foo
<Id>bar
<Id>foo
<Id>foo-1
However, I would like to achieve this kind of output without having to write file to disk in each step. For thousands of the string occurrences it takes a lot of time. Any idea how to do it?
=========================
Edit 2023-02-16
I tried the solution from zett42 and it works fine with simple XML structure. In my case there is some complication which was not important in my text processing approach.
Root and some other elements names in the structure of processed XML file contain colon and there must be some special setting for "-XPath" for this situation. Or, maybe the solution is outside of Powershell scope.
<?xml version='1.0' encoding='UTF-8'?>
<C23A:SC777a xmlns="urn:C23A:xsd:$SC777a" xmlns:C23A="urn:C23A:xsd:$SC777a" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:C23A:xsd:$SC777a SC777a.xsd">
<C23A:FIToDDD xmlns="urn:iso:std:iso:20022:tech:xsd:pacs.008.001.02">
<CxAAA>
<DxBBB>
<ABC>
<Id>ZZZZZZ999999</Id>
</ABC>
</DxBBB>
<CxxCCC>
<ABC>
<Id>ABC123456789</Id>
</ABC>
</CxxCCC>
</CxAAA>
<CxAAA>
<DxBBB>
<ABC>
<Id>ZZZZZZ999999</Id>
</ABC>
</DxBBB>
<CxxCCC>
<ABC>
<Id>ABC123456789</Id>
</ABC>
</CxxCCC>
</CxAAA>
</C23A:FIToDDD>
<C23A:PmtRtr xmlns="urn:iso:std:iso:20022:tech:xsd:pacs.004.001.02">
<GrpHdr>
<TtREEE Abc="XV">123.45</TtREEE>
<SttlmInf>
<STTm>ABCA</STTm>
<CLss>
<PRta>SIII</PRta>
</CLss>
</SttlmInf>
</GrpHdr>
<TxInf>
<OrgnlTxRef>
<DxBBB>
<ABC>
<Id>YYYYYY888888</Id>
</ABC>
</DxBBB>
<CxxCCC>
<ABC>
<Id>ABC123456789</Id>
</ABC>
</CxxCCC>
</OrgnlTxRef>
</TxInf>
</C23A:PmtRtr>
</C23A:SC777a>
As commented, it is not recommended to process XML like a text file. This is a brittle approach that depends too much on the formatting of the XML. Instead, use a proper XML parser to load the XML and then process its elements in an object-oriented way.
# Use XmlDocument (alias [xml]) to load the XML
$xml = [xml]::new(); $xml.Load(( Convert-Path -LiteralPath input.xml ))
# Define the ID replacements
$searchString = 'ABC123456789'
$replacements = 'foo','bar','baz','foo-1','bar-1'
# Process the text of all ID elements that match the search string, regardless how deeply nested they are.
$xml | Select-Xml -XPath '//Id/text()' | ForEach-Object Node |
Where-Object Value -eq $searchString | ForEach-Object {
# Replace the text of the current element by a randomly choosen string
$_.Value = Get-Random $replacements
}
# Save the modified document to a file
$xml.Save( (New-Item output.xml -Force).Fullname )
$xml | Select-Xml -XPath '//Id/text()' selects the text nodes of all Id elements, regardless how deeply nested they are in the XML DOM, using the versatile Select-Xml command. The XML nodes are selected by specifying an XPath expression.
Regarding your edit, when you have to deal with XML namespaces, use the parameter -Namespace to specify a namespace prefix to use in the XPath expression for the given namespace URI. In this example I've simply choosen a as the namespace prefix:
$xml | Select-Xml -XPath '//a:Id/text()' -Namespace #{a = 'urn:iso:std:iso:20022:tech:xsd:pacs.008.001.02'}
ForEach-Object Node selects the Node property from each result of Select-Xml. This simplifies the following code.
Where-Object Value -eq $searchString selects the text nodes that match the search string.
Within ForEach-Object, the variable $_ stands for the current text node. Assign to its Value property to change the text.
The Convert-Path and New-Item calls make it possible to use a relative PowerShell path (PSPath) with the .NET XmlDocument class. In general .NET APIs don't know anything about the current directory of PowerShell, so we have to convert the paths before passing to .NET API.
Apologies if this is irrelevant but I'm new to powershell and I've been scratching my head on this for a few days on and off now. I'm trying to write a script that will output two columns of data to a html document. I've achieved most of it by learning through forums and testing different combinations.
The problem is although it gives me the result I need within powershell itself; it will not properly display the second column results for Net Log Level.
So the script looks at some folders and pulls the * value which is always three digits (this is the Site array). It then looks within each of these folders to the Output folder and grabs a Net Log Level node from a file inside there. The script is correctly listing the Sites but is only showing the last value for Net Log Level which is 2. You can see this in the screenshot above. I need this to take every value for each Site and display as appropriate. The image of the incorrect result is below. I need the result to be 1,4,2,2,2. Any help would be greatly appreciated!
function getSite {
Get-ChildItem C:\Scripts\ServiceInstalls\*\Output\'Config.exe.config' | foreach {
$Site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content $_
$NetLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']")
$NetLogLevel = $NetLogLevel.value
New-Object -TypeName System.Collections.ArrayList
$List1 += #([System.Collections.ArrayList]#($Site))
New-Object -TypeName System.Collections.ArrayList
$List2 += #([System.Collections.ArrayList]#($NetLogLevel))
}
$Results = #()
ForEach($Site in $List1){
$Results += [pscustomobject]#{
"Site ID" = $Site
"Net Log Level" = $NetLogLevel
}
}
$Results | ConvertTo-HTML -Property 'Site','Net Log Level' | Set-Content Output.html
Invoke-Item "Output.html"
}
getSite
Restructure your code as follows:
Get-ChildItem 'C:\Scripts\ServiceInstalls\*\Output\Config.exe.config' |
ForEach-Object {
$site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content -Raw $_.FullName
$netLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']").InnerText
# Construct *and output* a custom object for the file at hand.
[pscustomobject] #{
'Site ID' = $site
'Net Log Level' = $netLogLevel
}
} | # Pipe the stream of custom objects directly to ConvertTo-Html
ConvertTo-Html | # No need to specify -Property if you want to use all properties.
Set-Content Output.html
As for what you tried:
New-Object -TypeName System.Collections.ArrayList in effect does nothing: it creates an array-list instance but doesn't save it in a variable, causing it to be enumerated to the pipeline, and since there is nothing to enumerate, nothing happens.
There is no point in wrapping a [System.Collections.ArrayList] instance in #(...): its elements are enumerated and then collected in a regular [object[]] array - just use #(...) by itself.
Using += to "grow" an array is quite inefficient, because a new array must be allocated behind the scenes every time; often there is no need to explicitly create an array - e.g. if you can simply stream objects to another command via the pipeline, as shown above, or you can let PowerShell itself implicitly create an array for you by assigning the result of a pipeline or foreach loop as a whole to a variable - see this answer.
Also note that when you use +=, the result is invariably a regular [object[] array, even if the RHS is a different collection type such as ArrayList.
There are still cases where iteratively creating an array-like collection is necessary, but you then need to use the .Add() method of such a collection type in order to grow the collection efficiently - see this answer.
Instead of populating two separate lists, simply create the resulting objects in the first loop:
function getSite {
$Results = Get-ChildItem C:\Scripts\ServiceInstalls\*\Output\'Config.exe.config' | ForEach-Object {
$Site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content $_
$NetLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']")
$NetLogLevel = $NetLogLevel.value
[pscustomobject]#{
"Site ID" = $Site
"Net Log Level" = $NetLogLevel
}
}
$Results | ConvertTo-HTML -Property 'Site', 'Net Log Level' | Set-Content Output.html
Invoke-Item "Output.html"
}
getSite
I need to sort specific value of BuildName: from web request response.
How can I store a particular response in a variable?
Trying with following command
Invoke-WebRequest -Uri https://s3.amazonaws.com/$url/env.js | select-object Content
I am getting the following response. I need to sort the values from there
Response will be like this
window.env = {
// Hardcode environment variables in here but tokenize customer specific ones with #{}#
BuildName: 'AppClient-develop-0716.4'
GRAPH_QL_HOST: 'https://xyz.google.com/graphql'
};
From her need to get value of BuildName:
You can convert the content to individual strings and parse those to get the individual values.
Example
# $Content = (Invoke-WebRequest -Uri https://s3.amazonaws.com/$url/env.js).Content
$Content = #'
window.env = {
// Hardcode environment variables in here but tokenize customer specific ones with #{}#
BuildName: 'AppClient-develop-0716.4'
GRAPH_QL_HOST: 'xyz.google.com/graphql'
};
'#
$Content -split "`r`n" | Select-String "\w+:" | % { ($_ -split ": ")[1]}
I've encountered a situation that I cannot seem to find a solution to. I am scraping a website using Invoke-WebRequest and when I look at my output from my array, several of the properties are System.Objects. I need to find a way to have them be strings so that when I Export-Csv I can actually see the values. Here is my code:
$params = #{api_id='';api_key='';page_size='100';site_id=''}
$stats = Invoke-WebRequest https://my.incapsula.com/api/visits/v1 -Method Post -Body $params
$s = $stats
$s = $s | ConvertFrom-Json
$s = $s.visits
Here are what my results look like:
My solution was to create a new custom object and use the following syntax on the properties that were an object themselves:
($_ | select -expandproperty 'propertyname')
At the moment I am working on a script to automate a process in IE to add Computer Names and their MACs so we can image them. The page has two fields one for MAC and one for a computer name then a add new button. I had to come to a pretty sloppy solution for avoiding a popup from the page by just quitting out of the com object after submitting.
I don't have much experience with Powershell yet and none with working with CSVs so I'm having a bit of trouble making this work. My goal is to have the script read two entries from a row fill out the correct field then submit it then move to the next row and repeat.
Right now what it does is fills out the fields with undefined in both fields, then submits and repeats.
EDIT: I have edited my code slightly just so it confirms what is trying to read.This is what the results look like. I believe #WalterMitty is on to something that something is wrong with $ie.document.getElementsByName lines, I just tried $ie.document.getElementById but that didn't fill out any fields. It seems it has no problem reading the CSV, but it does have a problem entering the information it reads into the fields properly.
This is an example of what the CSV would look like.
NewComputerName,NewMACAddress
ComputerName1,111122223333
ComputerName2,112233446677
ComputerName3,AAAABBBBCCCC
ComputerName4,AABBCCDDEEFF
This is what my code currently looks like.
cls
$URL = ""
$iterator = 1;
$csv = Get-Content C:\example1.csv
foreach($row in $csv)
{
#starts IE
$ie = new-object -ComObject "InternetExplorer.Application"
$ie.visible = $true
$ie.navigate($URL)
while($ie.Busy -eq $true) { start-sleep -Milliseconds 100 }
($ie.document.getElementsByName("mc_id") |select -first 1).value = $_.NewComputerName;
($ie.document.getElementsByName("mac_id") |select -first 1).value = $_.NewMACAddress;
$ie.document.forms | Select -First 1| % { $_.submit() };
$ie.quit()
$iterator++
write-host "$iterator new ID(s) added"
write-host $row.NewComputerName - $row.NewMACAddress
}
$URL = ""
$iterator = 1
# use Import-Csv for CSV files
$csv = Import-Csv "C:\example1.csv" -Delimiter ","
foreach($row in $csv) {
Write-Host "$iterator new ID(s) added"
#starts IE
$ie = New-Object -ComObject "InternetExplorer.Application"
$ie.Visible = $true
$ie.Navigate($URL)
while ($ie.Busy -eq $true) { Start-Sleep -Milliseconds 100 }
# $_ is not defined in foreach blocks, you have to use $row here
($ie.Document.getElementsByName("mc_id") | Select-Object -First 1).Value = $row.NewComputerName
($ie.Document.getElementsByName("mac_id") | Select-Object -First 1).Value = $row.NewMACAddress
$ie.Document.Forms | Select-Object -First 1 | ForEach-Object { $_.submit() }
$ie.Quit()
$iterator++
}
I'm having similar issues but I just found this may help you guys - I don't have 50 reputation to comment sorry :/....
I messed around with the -Path of the Import-CSV command but I just couldn’t make it work. Apparently this has nothing to do with the path of the CSV file. The Warlock posted this on his blog:
Long story short, the error came from having trailing blank columns in
my CSV. Import-Csv uses the first row in the CSV as names for the
columns (unless you specify otherwise) and when you have blank columns
(or at least multiple blank columns) it causes this error as it
doesn’t have a valid name for them.
Instead of changing the file, I changed my import command to include the headers as per Dale's comment and it worked perfectly:
$data = import-csv "C:\Sharepoint.csv" -header("Department","AD Group","Members","Notes")
The Warlock and Dale saved me lots of time, please stop by the Warlock’s blog and give them a big Thanks
Consider using Import-Csv and Invoke-WebRequest in combination, e.g. like this:
import-csv .\example.csv | %{ iwr http://someurl.local -body #{mc_id=$_.NewComputerName; mac_id=$_.NewMACAddress} -Method POST }
It will read the csv file, iterate over the records and create a application/x-www-form-urlencoded POST request with the values from each record.
When you use iwr (Invoke-WebRequest) and pass a hash table as the "body" it will act as if it is a form being submitted. The POST method will submit the form values as application/x-www-form-urlencode. Without the POST method it would submit the form as if it was a GET, i.e. pass the values in the url.
If you need authentication, session support etc. then read the documentation for Invoke-WebRequest.
Using IE to automate web requests is brittle and error prone.