I've encountered a situation that I cannot seem to find a solution to. I am scraping a website using Invoke-WebRequest and when I look at my output from my array, several of the properties are System.Objects. I need to find a way to have them be strings so that when I Export-Csv I can actually see the values. Here is my code:
$params = #{api_id='';api_key='';page_size='100';site_id=''}
$stats = Invoke-WebRequest https://my.incapsula.com/api/visits/v1 -Method Post -Body $params
$s = $stats
$s = $s | ConvertFrom-Json
$s = $s.visits
Here are what my results look like:
My solution was to create a new custom object and use the following syntax on the properties that were an object themselves:
($_ | select -expandproperty 'propertyname')
Related
I have a param section of a Invoke-RestMethod that i have to keep as is, but also add parameters:
$param = #{
...
Body = '{"entry":[{"#name":SOMETEXT,"static":{"member":[ANOTHERTEXT]}}]}'
...
}
Since the entire Body is in single quotes, any $parameter added will be treated like a string instead of a normal PowerShell parameter
In this case SOMETEXT and ANOTHERTEXT will be $HubAddressObject and $ArrayList, respectivly.
How do i make that Body entry work with parameters, and keeping the same structure (this is part of a Panorama box)?
What i would need would be:
Body = '{"entry":[{"#name":$HubAddressObject,"static":{"member":[$ArrayList]}}]}'
Thanks.
I'd recommend using ConvertFrom-Json / ConvertTo-Json for that kind of thing.
To keep it one 1 line, you can use the -Compress switch.
$params = #{
Body = '{"entry":[{"#name":"SOMETEXT","static":{"member":"[ANOTHERTEXT]"}}]}'
}
# Create a PSObject representation of your JSON
$jsonObj = $Params.body | ConvertFrom-Json
#Modify whatever you want
$jsonObj.entry[0].'#name' = 'NewText'
# Convert the Object back to Json.
$Params.Body = $JsonObj | Convertto-Json -Compress -Depth 4
Json comparison
# Starting off Json
{"entry":[{"#name":"SOMETEXT","static":{"member":"[ANOTHERTEXT]"}}]}
# Modified JSON
{"entry":[{"#name":"NewText","static":{"member":"[ANOTHERTEXT]"}}]}
I am trying to genrate URLs using a bunch of strings and an anchor URL
strings in s.txt are
123
234
345
anchor URL is https://testurl.com/prod/hostdetails.php?qs=
The code I am using
$ur = gc C:\temp\s.txt
foreach($u in $ur) {
$test = invoke-webrequest -uri "https://testurl.com/prod/hostdetails.php?qs=$u" -UseDefaultCredentials
}
$test
but it returns data only for
https://testurl.com/prod/hostdetails.php?qs=345
Its because you reassign $test each time within your loop. I suggest this solution:
$test = 'https://testurl.com/prod/hostdetails.php?{0}' -f ((C:\temp\s.txt) -join '&')
You edited your question. This is probably what you are looking for. Please notice the += operator to concat the results:
$result = #()
Get-Content C:\temp\s.txt | ForEach-Object {
$result += invoke-webrequest -uri "https://testurl.com/prod/hostdetails.php?qs=$($_)"
}
I have a script which I'm trying to modify, which adds objects to a firewall via powershell, using Invoke-RestMethod
The current script has the following code;
#Import CSV and set variables
$csv = import-csv C:\Powershell\groups.csv
# RESTful API Call
$csv | ForEach-Object {
$Name = $_.name
$Member1 =$_.member1
$Member2 =$_.member2
Invoke-RestMethod -Uri https://172.16.16.16:4444/webconsole/APIController?reqxml=<Request><Login><Username>admin</Username><Password>password</Password></Login><Set%20operation=%27add%27><IPHostGroup><Name>$Name</Name><IPFamily>IPv4</IPFamily><HostList><Host>$Member1</Host><Host>$Member2</Host></IPHostGroup></Set></Request>
}
I am wanting to import the hostgroups via groups.csv which (in my test) has 3 columns as follows;
Name,Member1,Member2
TestGroup,TestHost1,TestHost2
TestGroup2,TestHost3,TestHost4
etc.
My problem is that in the real data, there are varying amount of hosts in each group, some have hundreds. I'm not sure how to get these into the command without defining a variable for each possible member. Even then, say I created $Member(s) all the way to 200 (be gentle, I'm not a real coder!) and then imported them in manually one by one in the Invoke-Restmethod command (Might as well do it by hand at that point!) I'm not sure the command would handle the blank inputs in the cases where there were only a few hosts in the group.
(In other words if my csv had the following entries;)
Name,Member1,Member2,Member3,Member4
TestGroup,TestHost1,TestHost2,TestHost3,TestHost4
TestGroup2,TestHost5,TestHost6
TestGroup3,TestHost7
And I did;
# RESTful API Call
$csv | ForEach-Object {
$Name = $_.name
$Member1 =$_.member1
$Member2 =$_.member2
$Member3 =$_.member3
$Member4 =$_.member4
The Rest call for the third group would end up running as;
Invoke-RestMethod -Uri https://172.16.16.16:4444/webconsole/APIController?reqxml=<Request><Login><Username>admin</Username><Password>password</Password></Login><Set%20operation=%27add%27><IPHostGroup><Name>TestGroup3</Name><IPFamily>IPv4</IPFamily><HostList><Host>TestHost7</Host><Host></Host><Host></Host><Host></Host></IPHostGroup></Set></Request>
Can anyone point me in the direction of a better way of doing this?
You can get all member names using .PSObject.Properties.Name
Example:
$Csv = Import-Csv -Path 'C:\Powershell\groups.csv'
# Request XML template
$RequestTpl = #'
<Request>
<Login>
<Username>admin</Username>
<Password>password</Password>
</Login>
<Set%20operation=%27add%27>
<IPHostGroup>
<Name>{0}</Name>
<IPFamily>IPv4</IPFamily>
<HostList>
{1}
</HostList>
</IPHostGroup>
</Set>
</Request>
'#
# Host list XML template
$RequestHostListTpl = '<Host>{0}</Host>'
$Csv | ForEach-Object {
<#
Get names of all the properties in the current object
Leave only those that don't match '^Name$' regex.
-match, when operates on collections, returns matched items
You can use
$_.PSObject.Properties.Name | Where-Object {$_ -ne 'Name'}
but it's a bit slower.
#>
$Members = #($_.PSObject.Properties.Name) -notmatch '^Name$'
# Build list of hosts
$RequestHostList = foreach ($item in $Members) {
# Only add item if it's not empty
if ($_.$item) {
$RequestHostListTpl -f $_.$item
}
}
# Build request XML
$Request = $RequestTpl -f $_.Name, -join $RequestHostList
# Remove newlines to make it one long string
$Request = $Request -replace '\r|\n'
# Show resulting url
"Invoke-RestMethod -Uri https://172.16.16.16:4444/webconsole/APIController?reqxml=$Request"
# Uncomment to actually invoke API call
#Invoke-RestMethod -Uri "https://172.16.16.16:4444/webconsole/APIController?reqxml=$Request"
}
I'm trying to build a script that will grab some text from a website and place it into an excel sheet, but I'm having trouble with using a variable in the URL, when using a variable my output is http://#{variable=xyz}.com, I'm trying to get the xyz only,
any help would be greatly appreciated
my code:
$ip = get-content c:\..
foreach ($ip in $ipadd){
$url = http://$ip/mywebsite.com
$html = invoke-webrequest -uri $url
$elements = $html.Allelements | where class -eq "class" | select -expandproperty innertext
}
I ended up using plain text and it worked just fine.
but id appreciate if anyone could tell me why I'm getting this http://#{variable=xyz}.com output from a csv file
I encountered a problem today while trying to get a value from a website using PowerShell.
This is the website.
I am trying to get the number "90" here, that in the webpage itself is the value of "Downloaded" (this number might be a little bigger when you look at it if there are more downloads):Screenshot of the element i am trying to return
<span title="Downloads" class="mod-card-info-tag" data-reactid=".0.0.0.2.0.0.2.2">
<div class="mod-card-info-tag-label" data-reactid=".0.0.0.2.0.0.2.2.0">90</div>
This is the PowerShell code i used to try and get the number "90" from the element above (i know i should use ".innertext" in the end, i just used the get-member to see if any object was found):
$URI = "https://mods.factorio.com/mods/TpTheGreat/TpTheGreats%20Large%20Roboport%20Logistics%20Area"
$HTML = Invoke-WebRequest -Uri $URI
($HTML.ParsedHtml.getElementsByTagName("div") | Where{ $_.className -eq ‘mod-card-info-tag-label’ }) | Get-Member
When calling for the element by tag name like in my code above, I get an empty object.
I tried lots of things without any success.
It'll be really great if any of you could take a look and check if you are able to solve my problem.
Thanks a lot!!!
How about another approach:
$URI = "https://mods.factorio.com/mods/TpTheGreat/TpTheGreats%20Large%20Roboport%20Logistics%20Area"
$HTML = Invoke-WebRequest -Uri $URI
$arr = $HTML.AllElements.Item(9).innerHTML -split ' = '
$myObj = $arr[1].replace("`n"," ")
$myObj = $myObj.replace(";","") | ConvertFrom-Json
$myObj.mod.mod.downloads_count