I want to capture this 'text' shown int he picture below on a loop every minute, as this text changes every few minutes.
Here's the code I am using, and using HtmlAgilityPack.HtmlDocument
$metro = 'greatesthits'
$URL = "https://triplem.scadigital.com.au/stations/$metro/live"
[Reflection.Assembly]::LoadFile("C:\Users\makean\Downloads\htmlagilitypack.1.8.10\lib\Net45\HtmlAgilityPack.dll")
[HtmlAgilityPack.HtmlWeb]$web = #{}
[HtmlAgilityPack.HtmlDocument]$doc = $web.Load($url)
$doc.DocumentNode.SelectNodes(".//*[contains(#class,'sc-bdVaJa iHZvIS')]")
This is slimier code below, does the same thing, however just a different way of doing it
$metro = 'greatesthits'
$URL = "https://triplem.scadigital.com.au/stations/$metro/live"
Add-Type -path 'C:\Users\makean\Downloads\htmlagilitypack.1.8.10\lib\Net45\HtmlAgilityPack.dll'
$doc = New-Object HtmlAgilityPack.HtmlDocument
$wc = New-Object System.Net.WebClient
$doc.LoadHtml($wc.DownloadString($url))
$doc.DocumentNode.SelectNodes(".//*[contains(#class,'sc-bdVaJa iHZvIS')]")
This class sc-bdVaJa iHZvIS is a div and sits just a bit higher than PlayerNowPlaying__TrackInfo-kia103-1 gDXfGh and PlayerNowPlaying__TrackInfo-kia103-1 gDXfGh is what I want to capture, however when using this in my code, it returns blank.
How can I return just the text I want? Any help greatly appreciated.
I looked further at the thanks to the person above who pointed me in the right direction, checked the network option in Chrome 'inspect'. Grabbed the metadata from the stream URL.
$metro = '2classicrock'
$URL = 'https://wz2web.scahw.com.au/live/' + $metro + '_32.stream/playlist.m3u8'
$null = (Invoke-WebRequest -Uri $URL).RawContent -match '(https.*m3u8.*)'
$StreamURL = $Matches[0]
$streamMetaData = Invoke-WebRequest -Uri $StreamURL
$null = $streamMetaData.RawContent -match '#EXTINF:4.*?,(.*)'
$Matches[1]
Related
Code below to download a file from website:
https://www.mcafee.com/enterprise/en-us/downloads/security-updates.html
File being downloaded
However, the file name is changing every day. For example: 'mediumepo4981dat.zip' today and 'mediumepo4980dat.zip' yesterday.
How can I create a script which I can run daily which works dynamically?
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$uri = "https://download.nai.com/products/datfiles/med/mediumepo4981dat.zip"
$filename = "C:\DownloadTest\mediumepo4981dat.zip"
$wc = New-Object System.Net.WebClient
$wc.UseDefaultCredentials = $true
$wc.DownloadFile($uri, $filename)
Please let me know if you need any further clarification.
Note: Looking for a solution without Invoke-WebRequest as that does not work in my current environment.
Seems like we can assume that the last file listed is always the newest, can't confirm if this will always be true it may change at some point. For the time being, this is how you can get the last file:
$uri = [uri] "https://download.nai.com/products/datfiles/med"
$wr = Invoke-WebRequest $uri
$file = $wr.ParsedHtml.getElementsByTagName('a') |
Select-Object -Last 1 -ExpandProperty TextContent
$toDownload = [uri]::new($uri, $file)
Now you can combine this with the rest of your script:
$filename = Join-Path 'C:\DownloadTest\' -ChildPath $file
$wc = New-Object System.Net.WebClient
$wc.UseDefaultCredentials = $true
$wc.DownloadFile($toDownload.AbsoluteUri, $filename)
Note that this will work only in Windows PowerShell.
$uri = [uri] "https://download.nai.com/products/datfiles/med"
$wr = Invoke-WebRequest $uri
$file = $wr.ParsedHtml.getElementsByTagName('a') |
Select-Object -Last 1 -ExpandProperty TextContent
$toDownload = [uri]::new($uri, $file)
is missing trailing '/' in $uri.
It should be
$uri = [uri] "https://download.nai.com/products/datfiles/med/"
Hello all, I've been able to get this code to get the latest download link for Nvidia Driver for a Quadro P1000.
I was wondering, if there's a better way to do this.
Here's my code:
#Get Nvidia Drivers
#This next section is JUST TO get the LINK to the LATEST driver from Nvidia
#WebContent gets the links that are "href" from the nvidia JS request. No idea if there's an easier wey to get this.
$Webcontent = (Invoke-WebRequest 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0').links.href
#The following line uses Regex regular expressions to MATCH and RETRIEVE ONE single value from the list of values in the previous line.
$NVIDIALinkToLatestDriver = [regex]::Match($Webcontent, '//(www.nvidia.com/download/driverResults.aspx/[0-9]*/en-us)').Groups[1].Value
#Link after the previous crap
$NVIDIADLPage = New-Object -COM "HTMLFile" #Creates a COM Object for easier search of the link.
[string]$htmlBody = (Invoke-WebRequest -Uri $NVIDIALinkToLatestDriver -UseBasicParsing).Content #Parses the content of the landing page to then look by id
$NVIDIADLPage.write([ref]$htmlBody)
$replace = [regex]::Replace($NVIDIADLPage.getElementById('lnkDwnldBtn').href, 'about:', 'www.nvidia.com') #Replaces the "about" with "www.nvidia.com"
$Webcontent = (Invoke-WebRequest $replace) #Replace Webcontent with the latest link.
[String]$NvidiaLinkToExe = $Webcontent.links.href -match ".*.exe$" #On this link there's the exe file for Nvidia Drivers
$NvidiaLinkToExe = $NvidiaLinkToExe -replace "^", "http:" #Replace start of line with the correct Nvidia Link.
Remove-Variable -Name NVIDIADLPage, Webcontent, replace -Force #cleanup of the previous mess.
if ($NvidiaLinkToExe -match 'http:\/\/.*[0-9]{2}\/(([0-9]{3}\.[0-9]{2}).*\.exe)') {
$NVIDIAExeToDownload = [PSCustomObject]#{
Url = $Matches[0];
Name = $Matches[1];
Version = $Matches[2]
}
}
As you can see, I have to create three Invoke-WebRequest just to get one link.
And, I think I made no use of piping, because I was unable to make it work.
Thanks!
Your code seems better than mine. I had to call Invoke-WebRequest 3 times to get the link as well and once more to download it.
$destination = 'C:\Temp'
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$pattern = "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match $pattern){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
if($content.RawContent -match $pattern){
Invoke-WebRequest "https://$($Matches.Package)" -OutFile (Join-Path $destination $($Matches.package -replace '.+/'))
}
}
If you just want the link and other info you can drop the fourth.
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
$download.URL = "https://$($Matches.Package)"
$download
}
Currently, I make a POST request to a external website then I am supposed to get a zip file in return. I can get the zip file, but it comes in an xml with just the name.zip and nothing is downloaded. I have no idea why it is not downloading. My code is below on the piece where I make the actual request. I am not sure if I am over engineering this or what else I would have to do to get the actual file to download.
$url = "https://thewebsite.net/v6_1?id=$messageID"
Write-Output($url)
$Body = [byte[]][char[]]$xmlMessage
Write-Output($Body)
$Request = [System.Net.HttpWebRequest]::CreateHttp($url);
$Request.Method="POST"
$Request.ContentType = 'text/xml;charset=utf-8'
$Request.ContentLength = $Body.Length
$Request.ClientCertificates.Add($Certificate)
Write-Output($Request.ClientCertificates)
$Stream = $Request.GetRequestStream();
$Stream.Write($Body, 0, $Body.Length);
$Response = $Request.GetResponse()
$totalLength = [System.Math]::Floor($Response.get_ContentLength()/1024)
$responseStream = $Response.GetResponseStream()
$targetStream = New-Object -TypeName System.IO.FileStream -ArgumentList "D:\path\to\save\test.txt", Create
$buffer = new-object byte[] 1GB
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $count
while ($count -gt 0)
{
[System.Console]::CursorLeft = 0
[System.Console]::Write("Downloaded {0}K of {1}K", [System.Math]::Floor($downloadedBytes/1024), $totalLength)
$targetStream.Write($buffer, 0, $count)
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $downloadedBytes + $count
Write-Output($count)
}
$targetStream.Flush()
$targetStream.Close()
$targetStream.Dispose()
$responseStream.Dispose()
Unfortunately without certain download URI it's hard to clarify either you case is nontrivial or you just select non optimal way to get remote file. Routine way to get ".zip" (or any other 'octet/stream' file) with Power-Shell is execute the following command
Invoke-WebRequest -uri "https://thewebsite.net/v6_1?id=$messageID" -Method "GET" -Outfile (-join($messageID,".zip"))
then $messageID.zip file would be created in directory from which you execute Power-Shell
Progress would be shown in console window automatically. I test this example just before write the answer and it works independently on method "POST"/"GET" when remote host actually return "octet/stream" in the response. Maybe in you case file is not directly returned after requesting
thewebsite.net/v6_1?id=$messageID
but it is not a point of you original question.
Have you tried using Invoke-WebRequest?
$path = [Environment]::GetFolderPath("MyDocuments")
Invoke-WebRequest "example.com" -OutFile "$path\ZippedFile.zip"
A variable does not have to be used, as the path can be completely defined in the Invoke-WebRequest line if desired.
I'm simply trying to do a HTTP POST request with some keys and values. I can't get this to work for the life of me and yes I know this should be simple.
Here's what I've tried:
$Body = [byte[]][char[]]'username=asdf';
$Request = [System.Net.HttpWebRequest]::CreateHttp('http://mysite/test.php');
$Request.Method = 'POST';
$Stream = $Request.GetRequestStream();
$Stream.Write($Body, 0, $Body.Length);
$Request.GetResponse();
This doesn't work in Powershell v2.0 because I get the error
Method
invocation failed because [System.Net.HttpWebRequest] doesn't contain
a method named 'CreateHttp'.
Next, I've taken someone else's example of:
$URI1 = "http://mysite/test.php"
$request = [System.Net.WebRequest]::Create($URI1)
$request.ContentType = "application/xml"
$request.Method = "POST"
$body = "username=test"
# $request | Get-Member for a list of methods and properties
try
{
$requestStream = $request.GetRequestStream()
$streamWriter = New-Object System.IO.StreamWriter($requestStream)
$streamWriter.Write($body)
}
finally
{
if ($null -ne $streamWriter) { $streamWriter.Dispose() }
if ($null -ne $requestStream) { $requestStream.Dispose() }
}
$res = $request.GetResponse()
but for some reason "username" doesn't get noticed when test.php echos $_POST['username']
Can someone please help tell me what I'm missing here? I've been googling for hours and everything I try isn't working for some reason. Works fine on Powershell versions greater than 2.0, but not 2.0 (default in Windows 7).
Invoke-WebRequest and Invoke-RestMethod do not work on Powershell v2.0, so I'm forced to find all these annoying alternatives.
* EDIT *
I got it working after finding another HTTP POST request example:
$url = "http://mysite/test.php"
$postData = "username=test"
$buffer = [text.encoding]::ascii.getbytes($postData)
[net.httpWebRequest] $req = [net.webRequest]::create($url)
$req.method = "POST"
$req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
$req.Headers.Add("Accept-Language: en-US")
$req.Headers.Add("Accept-Encoding: gzip,deflate")
$req.Headers.Add("Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7")
$req.AllowAutoRedirect = $false
$req.ContentType = "application/x-www-form-urlencoded"
$req.ContentLength = $buffer.length
$req.TimeOut = 50000
$req.KeepAlive = $true
$req.Headers.Add("Keep-Alive: 300");
$reqst = $req.getRequestStream()
$reqst.write($buffer, 0, $buffer.length)
$reqst.flush()
$reqst.close()
[net.httpWebResponse] $res = $req.getResponse()
$resst = $res.getResponseStream()
$sr = new-object IO.StreamReader($resst)
$result = $sr.ReadToEnd()
$res.close()
This was discovered on another site before additional comments came here; however, I've tried the solutions from people's suggestions below and also was able to get this working.
In the first example code, the method should be named Create, Not CreateHttp
In the second block of code, you set the content-type to 'application/xml', but the body is plain-text.
This method, and your initial example itself, would work in a newer version of PowerShell. Try installing dotnet 4.5 on your system, then WMF 4.0, and this should work with no issue.
The dotnet class of [System.Net.HttpWebRequest] didn't have the static method CreateHttp() until dotnet 4.5, as seen here on MSDN Docs.
Why not just update this one machine to a newer version of PowerShell? It will be a LOT less pain.
I'm trying to write a script that checks my current public IP address to one that I've written in to see if it has changed. So far I've got this:
$NIP = "97.121.63.227"
$url = "http://checkip.dyndns.com"
$webclient = New-Object System.Net.WebClient
$Ip = $webclient.DownloadString($url)
$Ip2 = $Ip.ToString()
$ip3 = $Ip2.Split(" ")
$ip4 = $ip3[5]
$ip5 = $ip4.replace("</body>","")
$FinalIPAddress = $ip5.replace("</html>","")
$Prod = $NIP.CompareTo($FinalIPAddress)
$Prod
Whenever I run it though, comes up
Missing property name after reference operator.
At line:1 char:17
Pretty new to PowerShell and have no idea what I'm doing wrong
maybe you could use a regex to avoid multiple split and replace :
$NIP = "97.121.63.227"
$url = "http://checkip.dyndns.com"
$webclient = New-Object System.Net.WebClient
$Ip = $webclient.DownloadString($url)
$FinalIPAddress=[regex]::match($ip,"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b").Value
$NIP -eq $FinalIPAddress
I was not able to reproduce the error unfortunately. Perhaps set a breakpoint (F9) at the top line and step through (F10) the script to narrow in on the cause. The error message is pointing to line 1...did you include that line in your example?