I encountered a problem today while trying to get a value from a website using PowerShell.
This is the website.
I am trying to get the number "90" here, that in the webpage itself is the value of "Downloaded" (this number might be a little bigger when you look at it if there are more downloads):Screenshot of the element i am trying to return
<span title="Downloads" class="mod-card-info-tag" data-reactid=".0.0.0.2.0.0.2.2">
<div class="mod-card-info-tag-label" data-reactid=".0.0.0.2.0.0.2.2.0">90</div>
This is the PowerShell code i used to try and get the number "90" from the element above (i know i should use ".innertext" in the end, i just used the get-member to see if any object was found):
$URI = "https://mods.factorio.com/mods/TpTheGreat/TpTheGreats%20Large%20Roboport%20Logistics%20Area"
$HTML = Invoke-WebRequest -Uri $URI
($HTML.ParsedHtml.getElementsByTagName("div") | Where{ $_.className -eq ‘mod-card-info-tag-label’ }) | Get-Member
When calling for the element by tag name like in my code above, I get an empty object.
I tried lots of things without any success.
It'll be really great if any of you could take a look and check if you are able to solve my problem.
Thanks a lot!!!
How about another approach:
$URI = "https://mods.factorio.com/mods/TpTheGreat/TpTheGreats%20Large%20Roboport%20Logistics%20Area"
$HTML = Invoke-WebRequest -Uri $URI
$arr = $HTML.AllElements.Item(9).innerHTML -split ' = '
$myObj = $arr[1].replace("`n"," ")
$myObj = $myObj.replace(";","") | ConvertFrom-Json
$myObj.mod.mod.downloads_count
Related
I am trying to pass a MAC Address and code (1,2,3,4) to Invoke-WebRequest.
Manually the command is working fine, but I am not able to do it via command.
The manual command that works is:
Invoke-WebRequest -Uri https://mywebsite.org/pcconfig/setstate.php?mac=F832E3A2503B"&"state=4
Now when I break this up into variable to use with the mac from the machine I do the following.
$LiveMAC = Get-NetAdapter -Physical |
where Status -eq "Up" |
Select-Object -ExpandProperty PermanentAddress
$Str1 = "https://mywebsite.org/pcconfig/setstate.php?mac=$LiveMAC"
$Str2 = $Str1 + '"&"' + 'state=4'
Invoke-WebRequest -Uri $Str2
When I run the above code I do not see errors in red, it seems to process but does not work.
Looking at $Str2 I see the below output, which seems correct, but when passing it as above it fails to work.
https://mywebsite.org/pcconfig/setstate.php?mac=F832E3A2503B"&"state=4
The double quotes in a statement like
Invoke-WebRequest -Uri https://example.org/some/sit.php?foo=x"&"bar=y
mask the ampersand for PowerShell, because otherwise otherwise PowerShell would throw an error that a bare & is reserved for future use. A more canonical way of avoiding this is to put the entire URI in quotes:
Invoke-WebRequest -Uri "https://example.org/some/sit.php?foo=x&bar=y"
Either way the actual URI that is passed to Invoke-WebRequest is
https://example.org/some/sit.php?foo=x&bar=y
without the quotes.
However, in a code snippet like this:
$Str1 = "https://example.org/some/site.php?foo=$foo"
$Str2 = $Str1 + '"&"' + 'state=4'
you're including literal double quotes as part of the URI, so the actual URI passed to the cmdlet would be
https://example.org/some/sit.php?foo=x"&"bar=y
which is invalid.
With that said, you don't need string concatenation for building your URI anyway. Simply put your variable in the double-quoted string:
$uri = "https://example.org/some/sit.php?foo=${foo}&bar=y"
If you need to insert the value of an object property or array element use either a subexpression
$uri = "https://example.org/some/sit.php?foo=$($foo[2])&bar=y"
or the format operator
$uri = 'https://example.org/some/sit.php?foo={0}&bar=y' -f $foo[2]
I am trying to genrate URLs using a bunch of strings and an anchor URL
strings in s.txt are
123
234
345
anchor URL is https://testurl.com/prod/hostdetails.php?qs=
The code I am using
$ur = gc C:\temp\s.txt
foreach($u in $ur) {
$test = invoke-webrequest -uri "https://testurl.com/prod/hostdetails.php?qs=$u" -UseDefaultCredentials
}
$test
but it returns data only for
https://testurl.com/prod/hostdetails.php?qs=345
Its because you reassign $test each time within your loop. I suggest this solution:
$test = 'https://testurl.com/prod/hostdetails.php?{0}' -f ((C:\temp\s.txt) -join '&')
You edited your question. This is probably what you are looking for. Please notice the += operator to concat the results:
$result = #()
Get-Content C:\temp\s.txt | ForEach-Object {
$result += invoke-webrequest -uri "https://testurl.com/prod/hostdetails.php?qs=$($_)"
}
Below is the HTML DOM, I want to store DatePlannedStartTimeStamp value "01/24/2017 18:00" in a variable, print and call later
<TD class=dbData><SPAN onmouseover="ShowDay(this.innerHTML, 231320)"
onmouseout=nd() id=DatePlannedStartTimeStamp>01/24/2017 18:00</SPAN></TD>
I tried below but it does not return any value
$HTML = Invoke-WebRequest -Uri $URL -UseDefaultCredential
$WebPageText = ($HTML.ParsedHtml.getElementsByTagName("a") | Where-
Object{$_.ID -match "DatePlannedStartTimeStamp"}).innerText
echo $WebPageText
Not sure if my tagname is correct. I'm still learning powershell script, it would be great help if there is a way to it.
I'm trying to build a script that will grab some text from a website and place it into an excel sheet, but I'm having trouble with using a variable in the URL, when using a variable my output is http://#{variable=xyz}.com, I'm trying to get the xyz only,
any help would be greatly appreciated
my code:
$ip = get-content c:\..
foreach ($ip in $ipadd){
$url = http://$ip/mywebsite.com
$html = invoke-webrequest -uri $url
$elements = $html.Allelements | where class -eq "class" | select -expandproperty innertext
}
I ended up using plain text and it worked just fine.
but id appreciate if anyone could tell me why I'm getting this http://#{variable=xyz}.com output from a csv file
I've encountered a situation that I cannot seem to find a solution to. I am scraping a website using Invoke-WebRequest and when I look at my output from my array, several of the properties are System.Objects. I need to find a way to have them be strings so that when I Export-Csv I can actually see the values. Here is my code:
$params = #{api_id='';api_key='';page_size='100';site_id=''}
$stats = Invoke-WebRequest https://my.incapsula.com/api/visits/v1 -Method Post -Body $params
$s = $stats
$s = $s | ConvertFrom-Json
$s = $s.visits
Here are what my results look like:
My solution was to create a new custom object and use the following syntax on the properties that were an object themselves:
($_ | select -expandproperty 'propertyname')