Code below to download a file from website:
https://www.mcafee.com/enterprise/en-us/downloads/security-updates.html
File being downloaded
However, the file name is changing every day. For example: 'mediumepo4981dat.zip' today and 'mediumepo4980dat.zip' yesterday.
How can I create a script which I can run daily which works dynamically?
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$uri = "https://download.nai.com/products/datfiles/med/mediumepo4981dat.zip"
$filename = "C:\DownloadTest\mediumepo4981dat.zip"
$wc = New-Object System.Net.WebClient
$wc.UseDefaultCredentials = $true
$wc.DownloadFile($uri, $filename)
Please let me know if you need any further clarification.
Note: Looking for a solution without Invoke-WebRequest as that does not work in my current environment.
Seems like we can assume that the last file listed is always the newest, can't confirm if this will always be true it may change at some point. For the time being, this is how you can get the last file:
$uri = [uri] "https://download.nai.com/products/datfiles/med"
$wr = Invoke-WebRequest $uri
$file = $wr.ParsedHtml.getElementsByTagName('a') |
Select-Object -Last 1 -ExpandProperty TextContent
$toDownload = [uri]::new($uri, $file)
Now you can combine this with the rest of your script:
$filename = Join-Path 'C:\DownloadTest\' -ChildPath $file
$wc = New-Object System.Net.WebClient
$wc.UseDefaultCredentials = $true
$wc.DownloadFile($toDownload.AbsoluteUri, $filename)
Note that this will work only in Windows PowerShell.
$uri = [uri] "https://download.nai.com/products/datfiles/med"
$wr = Invoke-WebRequest $uri
$file = $wr.ParsedHtml.getElementsByTagName('a') |
Select-Object -Last 1 -ExpandProperty TextContent
$toDownload = [uri]::new($uri, $file)
is missing trailing '/' in $uri.
It should be
$uri = [uri] "https://download.nai.com/products/datfiles/med/"
Related
Hello all, I've been able to get this code to get the latest download link for Nvidia Driver for a Quadro P1000.
I was wondering, if there's a better way to do this.
Here's my code:
#Get Nvidia Drivers
#This next section is JUST TO get the LINK to the LATEST driver from Nvidia
#WebContent gets the links that are "href" from the nvidia JS request. No idea if there's an easier wey to get this.
$Webcontent = (Invoke-WebRequest 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0').links.href
#The following line uses Regex regular expressions to MATCH and RETRIEVE ONE single value from the list of values in the previous line.
$NVIDIALinkToLatestDriver = [regex]::Match($Webcontent, '//(www.nvidia.com/download/driverResults.aspx/[0-9]*/en-us)').Groups[1].Value
#Link after the previous crap
$NVIDIADLPage = New-Object -COM "HTMLFile" #Creates a COM Object for easier search of the link.
[string]$htmlBody = (Invoke-WebRequest -Uri $NVIDIALinkToLatestDriver -UseBasicParsing).Content #Parses the content of the landing page to then look by id
$NVIDIADLPage.write([ref]$htmlBody)
$replace = [regex]::Replace($NVIDIADLPage.getElementById('lnkDwnldBtn').href, 'about:', 'www.nvidia.com') #Replaces the "about" with "www.nvidia.com"
$Webcontent = (Invoke-WebRequest $replace) #Replace Webcontent with the latest link.
[String]$NvidiaLinkToExe = $Webcontent.links.href -match ".*.exe$" #On this link there's the exe file for Nvidia Drivers
$NvidiaLinkToExe = $NvidiaLinkToExe -replace "^", "http:" #Replace start of line with the correct Nvidia Link.
Remove-Variable -Name NVIDIADLPage, Webcontent, replace -Force #cleanup of the previous mess.
if ($NvidiaLinkToExe -match 'http:\/\/.*[0-9]{2}\/(([0-9]{3}\.[0-9]{2}).*\.exe)') {
$NVIDIAExeToDownload = [PSCustomObject]#{
Url = $Matches[0];
Name = $Matches[1];
Version = $Matches[2]
}
}
As you can see, I have to create three Invoke-WebRequest just to get one link.
And, I think I made no use of piping, because I was unable to make it work.
Thanks!
Your code seems better than mine. I had to call Invoke-WebRequest 3 times to get the link as well and once more to download it.
$destination = 'C:\Temp'
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$pattern = "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match $pattern){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
if($content.RawContent -match $pattern){
Invoke-WebRequest "https://$($Matches.Package)" -OutFile (Join-Path $destination $($Matches.package -replace '.+/'))
}
}
If you just want the link and other info you can drop the fourth.
$downloadlist = 'https://www.nvidia.com/Download/processFind.aspx?psid=73&pfid=842&osid=57&lid=1&whql=&lang=en-us&ctk=0&qnfslb=10&dtcid=0'
$content = Invoke-WebRequest $downloadlist -UseBasicParsing
$download = if($content.RawContent -match "(?s)<tr>.+?href='//(?<URL>.+?)'>.+?<td.+?>(?<Version>.+?)</td.+?td.+?>(?<Date>.+?)</td.+</tr>"){
[PSCustomObject]#{
URL = $Matches.URL
Version = $Matches.Version
Date = $Matches.Date
}
}
$pattern = '(?s)(?<Package>/content/driver[^"]+?{0}/.+?)(?=")' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest $download.url -UseBasicParsing
if($content.RawContent -match $pattern){
$pattern = '//(?<Package>.+?{0}.+exe)' -f ($download.Version -replace '.+\(|\)')
$content = Invoke-WebRequest -Uri "https://www.nvidia.com$($Matches.Package)" -UseBasicParsing
$download.URL = "https://$($Matches.Package)"
$download
}
My Powershell script works fine this way (it uses FTP to send IHM_OTP_CRT_20190812_0701.txt to the server and save it as newfile.txt)
$File = "Z:\Export\IHM_OTP_CRT_20190812_0701.txt"
$ftp = "ftp://ftpuse:mypass#e2b.kpsci.com/inbound/newfile.txt"
"ftp url: $ftp"
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
"Uploading $File..."
$webclient.UploadFile($uri, $File)
But it does not work if I change the second line
$ftp = "ftp://ftpuse:mypass#e2b.kpsci.com/inbound/newfile.txt"
to the following
$File = Get-ChildItem -Recurse | Where-Object { $_.Name -match 'IHM_OTP_CRT_.' } | sort LastWriteTime | select -last 1
$NewFileName = $File.Name
$ftp = "ftp://ftpuse:mypass#e2b.kpsci.com/inbound/" + $NewFileName
and this is driving me crazy.
I've tried various concatenation methods...I used the $var1``$var2 concept saved into $NewFileName to avoid the + sign, I've used parenthesis around the argument like:
$ftp = ("ftp://ftpuse:mypass#e2b.kpsci.com/inbound/" + $NewFileName)
And the more frustrating part is that when I use #echo, the concatenated string looks perfect. Also, this works fine:
$ftp = "ftp://ftpuse:mypass#e2b.kpsci.com/inbound/" + "IHM_OTP_CRT_20190812_0701.txt"
So, it's only concatenating with a separate object (even if it's a string) that doesn't work. I can concatenate to "blah" but not to a variable that equates to "blah". I've spent hours on this and I don't imagine it should be this difficult.
The error I am receiving is:
"Exception calling "UploadFile" with 2 argument(s): "The requested URI is invalid for this FTP command...at :17 char: 22..."
The error makes sense to me - it think's that my "concatenated" object contains a separate argument into the Upload File method, but I can't understand how to make it understand that I'm intending to pass a single string.
The issue is that the Get-ChildItem object .ToString() will only return the file name, not the full path. Therefore in the Webclient Upload you have to specify the .FullName property:
$webclient.UploadFile($uri, $File.FullName)
Full code:
$File = Get-ChildItem -Recurse | Where-Object { $_.Name -match 'IHM_OTP_CRT_.' } | sort LastWriteTime | select -last 1
$NewFileName = $File.Name
$ftp = "ftp://ftpuse:mypass#e2b.kpsci.com/inbound/" + $NewFileName
"ftp url: $ftp"
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
"Uploading $File..."
$webclient.UploadFile($uri, $File.FullName)
I want to capture this 'text' shown int he picture below on a loop every minute, as this text changes every few minutes.
Here's the code I am using, and using HtmlAgilityPack.HtmlDocument
$metro = 'greatesthits'
$URL = "https://triplem.scadigital.com.au/stations/$metro/live"
[Reflection.Assembly]::LoadFile("C:\Users\makean\Downloads\htmlagilitypack.1.8.10\lib\Net45\HtmlAgilityPack.dll")
[HtmlAgilityPack.HtmlWeb]$web = #{}
[HtmlAgilityPack.HtmlDocument]$doc = $web.Load($url)
$doc.DocumentNode.SelectNodes(".//*[contains(#class,'sc-bdVaJa iHZvIS')]")
This is slimier code below, does the same thing, however just a different way of doing it
$metro = 'greatesthits'
$URL = "https://triplem.scadigital.com.au/stations/$metro/live"
Add-Type -path 'C:\Users\makean\Downloads\htmlagilitypack.1.8.10\lib\Net45\HtmlAgilityPack.dll'
$doc = New-Object HtmlAgilityPack.HtmlDocument
$wc = New-Object System.Net.WebClient
$doc.LoadHtml($wc.DownloadString($url))
$doc.DocumentNode.SelectNodes(".//*[contains(#class,'sc-bdVaJa iHZvIS')]")
This class sc-bdVaJa iHZvIS is a div and sits just a bit higher than PlayerNowPlaying__TrackInfo-kia103-1 gDXfGh and PlayerNowPlaying__TrackInfo-kia103-1 gDXfGh is what I want to capture, however when using this in my code, it returns blank.
How can I return just the text I want? Any help greatly appreciated.
I looked further at the thanks to the person above who pointed me in the right direction, checked the network option in Chrome 'inspect'. Grabbed the metadata from the stream URL.
$metro = '2classicrock'
$URL = 'https://wz2web.scahw.com.au/live/' + $metro + '_32.stream/playlist.m3u8'
$null = (Invoke-WebRequest -Uri $URL).RawContent -match '(https.*m3u8.*)'
$StreamURL = $Matches[0]
$streamMetaData = Invoke-WebRequest -Uri $StreamURL
$null = $streamMetaData.RawContent -match '#EXTINF:4.*?,(.*)'
$Matches[1]
I need to download a file from a website every hour . Right now I have...
$url = "https://www.misoenergy.org/ria/Consolidated.aspx?format=csv"
$path = "C:\MISO.csv"
# param([string]$url, [string]$path)
if(!(Split-Path -parent $path) -or !(Test-Path -pathType Container (Split-Path -parent $path))) {
$path = Join-Path $pwd (Split-Path -leaf $path)
}
"Downloading [$url]`nSaving at [$path]"
$client = new-object System.Net.WebClient
$client.DownloadFile($url, $path)
#$client.DownloadData($url, $path)
$path
PAUSE
This is getting the response from the website and prompting me to open or save the file. I just want it to save the file. Thank you for any help.
Try this:
$url = "https://www.misoenergy.org/ria/Consolidated.aspx?format=csv"
$data = Invoke-WebRequest -Uri $url
$path = "C:\MISO.csv"
$data.content | Out-file $path
Of course there is no error handling, or checking. It assumes that Invoke-WebRequest has completed successfully and your endpoint does return raw CSV data.
Use wget in my opinion.
The following works for me.
$url = "https://www.misoenergy.org/ria/Consolidated.aspx?format=csv"
$path = "d:\my_cs_folder\MISO.csv"
wget -OutFile $path -Uri $url
I removed the checks for readablity. Destination file Path is changed too. Be advised that you may receive permission denied for placing your file on system drive's root.
I've been attempting a similar thing for a while now and it's very annoying as most cases will not work correctly and sending keystrokes to the save button requires the window to be active, at least for as far as I can see.
Due to this if the computer is being used it will not work, I still need to figure out a way past this but until now I've gotten it to work like this.
$IE = New-Object -ComObject InternetExplorer.Application
$IE.Visible = $true
$IE.Navigate($url)
$IEProc = Get-Process | Where-Object {$_.MainWindowHandle -eq $IE.HWND}
$WS = New-Object -ComObject WScript.Shell
$WS.AppActivate($IEProc.id)
Start-Sleep 2
[Windows.Forms.SendKeys]::SendWait('%{s}')
I'm aware of it being very ugly but it does the job :(
I want this script to choose the latest file from a folder and then send it via ftp to the server.
I think it is choosing the file late because there is a new file on the FTP after running it. However it crashes constantly showing
uploading .....
uploading .....
uploading .....
$Dir="C:/log1"
$ftp = "ftpftpftp"
$user = "useruseruser"
$pass = "passpasspass"
$latest = Get-ChildItem -Path $Dir | Sort-Object LastAccessTime -Descending | Select-Object -First 1
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)
for($latest){
"Uploading $latest..."
$uri = New-Object System.Uri($ftp+$latest.Name)
$webclient.UploadFile($uri, $latest.FullName)
}
I think you are using the wrong code block by accident. Currently you have created an infinite loop as you have no condition to how the for will exit.
A simple example of such a loop would be
for(){"Hello? Is it me you are looking for?"}
It should be structured like this
for (initialization; condition; repeat){code block}
an example would be
for($index =1; $index -lt 6;$index++){$index}
There is no need for that code block at all as long as $Dir is not empty. What you can do for a little error prevention is if($latest){} which will only work if $latest contains a file (in this code structure).
if($latest){
"Uploading $latest..."
$uri = New-Object System.Uri($ftp+$latest.Name)
$webclient.UploadFile($uri, $latest.FullName)
}
Your sample output does not have a file name in it so I suspect your $dir contains no files?