I would like to use PowerShell to download openBLAS from SourceForge. My problem is that the PS documentation I have read always refers to a specific file name, but, when I submit that request, SF redirects it to /download, and something happens, but I don't get what I am looking for. Here is the current code:
$url = 'https://sourceforge.net/projects/openblas/files/v0.3.6/OpenBLAS-0.3.6-x64.zip'
# $url = 'https://sourceforge.net/projects/openblas/files/v0.3.6/OpenBLAS-0.3.6-x64.zip/download'
$output = 'C:\BLAS\OpenBLAS-0.3.6-x64.zip'
Invoke-WebRequest $url -OutFile $output
It works for me in Windows 10 PowerShell 5.1.
Invoke-WebRequest -Uri $url -OutFile $path -UserAgent [Microsoft.PowerShell.Commands.PSUserAgent]::FireFox
source
Related
I'm trying to download a msi Package from a Page which does a redirection in the background which downloads the file I want.
This is the Page I want to download the msi from:
https://remotedesktopmanager.com/de/home/thankyou/rdmmsi
I tried several PowerShell Scripts, but none of them extracted the right download URL or downloaded the file.
With Invoke-Webrequest -Outfile C:xxx only the HTML is saved.
the page uses javascript with a timeout to redirect to the current setup file. that's why you cannot use Invoke-WebRequest as this uses the Internet Explorer engine under the hood which also performs this javascript window.location redirect (resulting in opening the url in the default browser).
To only get the raw HTML you have to use Invoke-RestMethod
$website = Invoke-RestMethod -Uri 'https://remotedesktopmanager.com/de/home/thankyou/rdmmsi'
The full website is now stored in the variable $website without interpreting the javascript.
To find the line with the string window.location i use Select-String which requires a file to be parsed. Thus the content of the variable is first stored in the file which is then parsed.
$tmpFilePath = 'C:\tmp\t.txt'
Out-File -FilePath $tmpFilePath -InputObject $website
$urlRedirectLine = Select-String -Path $tmpFilePath -SimpleMatch "window.location"
Remove-Item -Path $tmpFilePath
The new variable $urlRedirectLine (which content now is C:\tmp\t.txt:999: setTimeout(function () { window.location = 'https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2021.1.25.0.msi'; }, 4500);) contains the string we are looking for.
To extract that url i convert to variable to string and then use SubString() to extract the url itself. For this i look for the first ' and last ' in that variable.
$urlString = $urlRedirectLine.ToString()
$url = $urlString.Substring($urlString.IndexOf("'")+1,$urlString.LastIndexOf("'")-$urlString.IndexOf("'")-1)
Resulting in $url having the url https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2021.1.25.0.msi
To download the file you can again use Invoke-RestMethod
Invoke-RestMethod -Uri $url -OutFile 'C:\tmp\rdm.msi'
I was facing similar error while trying to download httpd.
Following worked for me (-UserAgent "NativeHost")
$ENV:HTTPD_DOWNLOAD_URL = "https://www.apachelounge.com/download/VS16/binaries/httpd-2.4.52-win64-VS16.zip"
$ENV:HTTPD_DOWNLOAD_ZIP = "httpd.zip"
Invoke-WebRequest -Uri $ENV:HTTPD_DOWNLOAD_URL -OutFile $ENV:HTTPD_DOWNLOAD_ZIP -UserAgent "NativeHost";
Reference
https://social.technet.microsoft.com/Forums/en-US/23fccc30-5f84-4a84-8160-c6e95102b11c/powershell-invokewebrequest-sourceforge-urlsredirectiondynamic-content?forum=winserverpowershell
The Link you give is of main download page which redirects after few second to the download.
The downloading link is "https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2021.1.25.0.msi"
Use the following command to download that MSI.
$url = "https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2021.1.25.0.msi"
$dest = "C:\Setup.RemoteDesktopManager.2021.1.25.0.msi"
Start-BitsTransfer -Source $url -Destination $dest
You can also use Invoke-WebRequest using the above variable
Invoke-WebRequest -Uri $url -OutFile $dest
Thank You
I have searched extensively for a solution but have yet to find success. I just want to be able to download files from my private GitHub repo using PowerShell. I want to use OAuth, not basic auth, so I have generated a token. But from here, none of the examples I've referenced worked for me. Best I could do was get a "Not Found" response.
An example of code I've tried is:
Invoke-WebRequest https://api.github.com/repos/MyAccount/MyRepo/contents/MyFile.txt -Headers #{"Authorization"="token 123456789012345678901234567890"} -OutFile C:\Temp\MyFile.txt
Result:
Invoke-WebRequest : {"message":"Not
Found","documentation_url":"https://docs.github.com/rest/reference/repos#get-repository-content"}
I'm fairly confident that I have the authentication right. I believe I just have the path wrong path to my file. Any help would be greatly appreciated.
Potential Duplicate use case relative to this SO discussion ...
PowerShell: retrieve file from GitHub
$url = 'https://github.com/mycompany/myrepo/blob/master/myscript.ps1'
$wc = New-Object -TypeName System.Net.WebClient
$wc.Headers.Add('Authorization','token your_token')
iex ($wc.DownloadString($url))
.. without Invoke-WebRequest of course.
See also:
Using PowerShell and oAuth
# Modified article code
Invoke-RestMethod https://api.github.com/repos/MyAccount/MyRepo/contents/MyFile.txt -Method Get -Headers #{"Authorization" = "Bearer $accessToken"}
I had to change the script in Powershell to get it working:
$credentials="<github_access_token>"
$repo = "<user_or_org>/<repo_name>"
$file = "<name_of_asset_file>"
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "token $credentials")
$headers.Add("Accept", "application/json")
$download = "https://raw.githubusercontent.com/$repo/main/$file"
Write-Host Dowloading latest release
Invoke-WebRequest -Uri $download -Headers $headers -OutFile $file
I am trying to download package from the below link using powershell.
https://www.tenable.com/downloads/nessus-agents
i do not have direct link for these package also when i click on download it ask to agree. I was able to do it on Linux using command shown below. Kindly advise how can i do it in windows.
"wget --no-check-certificate --post-data='accept="I accept the terms of this license"&x=""&sid=5mcia8gchg28attkc9oarah153&p=NessusAgent-7.4.2-amzn.x86_64.rpm' 'https://www.tenable.com/downloads/nessus-agents' -O NessusAgent-7.4.2-amzn.x86_64.rpm"
could not find anything tried option with invoke-webrequest
Invoke-RestMethod -Uri 'https://www.tenable.com/downloads/nessus-agents'
There's a GET query string parameter that indicates acceptance.
Simply add i_agree_to_tenable_license_agreement=true to your query string parameters.
Invoke-WebRequest -Uri 'https://www.tenable.com/downloads/api/v1/public/pages/nessus-agents/downloads/9762/download?i_agree_to_tenable_license_agreement=true' -OutFile 'NessusAgent-7.4.2-x64.msi'
You can easily get the IDs of the other files from their API endpoint like so:
(Invoke-WebRequest -Uri 'https://www.tenable.com/downloads/api/v1/public/pages/nessus-agents' | ConvertFrom-Json).downloads | Format-Table -AutoSize
This is similar syntax in Powershell, but it's just downloading a file with contents "OK".
$body = 'accept="I accept the terms of this license"&x=""&sid=5mcia8gchg28attkc9oarah153&p=NessusAgent-7.4.2-amzn.x86_64.rpm'
$uri = 'https://www.tenable.com/downloads/nessus-agents'
$resp = Invoke-WebRequest -Method Post -Body $body -Uri $uri -OutFile .\NessusAgent-7.4.2-amzn.x86_64.rpm
Maybe the "sid" variable needs to change per request.
I am trying to download file from the web using following command
Invoke-WebRequest $url -OutFile $filePath -Headers $Headers
I have argument, which contains this url and it is passed as parameter
[string]$artifactHttpAddress = $args[2]
Currently its value is
http://10.45.48.26/httpAuth/repository/downloadAll/TeamCityTest_Build/529:id/artifacts.zip
So, when I try to invoke WebRequest with following command
Invoke-WebRequest $artifactHttpAddress -OutFile c:/test.zip -Headers $Headers
it is downloading empty zip file .
but when I try to assign this url to the variable and invoke web request
$url = "http://10.45.48.26/httpAuth/repository/downloadAll/TeamCityTest_Build/529:id/artifacts.zip"
Invoke-WebRequest $url -OutFile c:/test.zip -Headers $Headers
It is working correctly, downloads zip file, which have some content in it.
I tried following script
Write-Host([string]$url -eq [string]$artifactHttpAddress)
Write-Host([string]$url)
Write-Host([string]$artifactHttpAddress)
It outputs
False
http://10.45.48.26/httpAuth/repository/downloadAll/TeamCityTest_Build/528:id/artifacts.zip
http://10.45.48.26/httpAuth/repository/downloadAll/TeamCityTest_Build/531:id/artifacts.zip
What is happening and why?
p.s. this script is inside ScriptBlock
It looks to me, based on your output, that $url and $artifactHttpAddress are not the same value. Does the ZIP file exist at the URL with 531 in it?
How can I use Invoke-WebRequest to download a file but automatically make the file name the same as if I downloaded via browser? I haven't found a way to make -OutFile work without manually specifying the file name. I'm fine with this involving a few other lines of code.
A good solution will:
Work even if the file name isn't in the request URL. For example, the URL to download the Visual Studio x64 Remote Debugging Tools is http://go.microsoft.com/fwlink/?LinkId=393217 but it downloads the file rtools_setup_x64.exe.
Not save the whole file to memory before writing to disk, unless that's what Invoke-WebRequest already does even with the -OutFile parameter (?)
Thanks!
For the example given you're going to need to get the redirected URL, which includes the file name to be downloaded. You can use the following function to do so:
Function Get-RedirectedUrl {
Param (
[Parameter(Mandatory=$true)]
[String]$URL
)
$request = [System.Net.WebRequest]::Create($url)
$request.AllowAutoRedirect=$false
$response=$request.GetResponse()
If ($response.StatusCode -eq "Found")
{
$response.GetResponseHeader("Location")
}
}
Then it's a matter of parsing the file name from the end of the responding URL (GetFileName from System.IO.Path will do that):
$FileName = [System.IO.Path]::GetFileName((Get-RedirectedUrl "http://go.microsoft.com/fwlink/?LinkId=393217"))
That will leave $FileName = rtools_setup_x64.exe and you should be able to download your file from there.
Try this method (may not always work because the file name may not be in the response header)
call Invoke-WebRequest to get the result. then you can inspect the result to look at what is in the headers.
get the file name from the response header (this could be in Headers.Location, or some other place. When I run my query for a url that i was troubleshooting, I found it in the Headers["Content-Disposition"] and it looks like inline; filename="zzzz.docx"
Create a new file based on the name and write the content to this file
Here is code sampe:
$result = Invoke-WebRequest -Method GET -Uri $url -Headers $headers
$contentDisposition = $result.Headers.'Content-Disposition'
$fileName = $contentDisposition.Split("=")[1].Replace("`"","")
$path = Join-Path $yourfoldername $fileName
$file = [System.IO.FileStream]::new($path, [System.IO.FileMode]::Create)
$file.write($result.Content, 0, $result.RawContentLength)
$file.close()
Thanks to Ryan I have a semi-usable function:
Function Get-Url {
param ( [parameter(position=0)]$uri )
invoke-webrequest -uri "$uri" -outfile $(split-path -path "$uri" -leaf)
}
A graphic file and xml file I have been able to download. When I try to download this webpage and open it with Edge it will work at times.
Essence of my code - this works.. using PS 5.1
I have commented out the normal -Outfile statement because that would imply I knew the filename in advance.
I have put this together based upon a number of sources, there are lots of ways of parsing the Content-Disposition headers so use whatever works for you.
$outpath = "C:\\temp\\"
The call:
$result = Invoke-WebRequest -method GET -Uri $resourceUrl -Headers
$resourceHeaders -Verbose #-OutFile $($outPath+"$nodeId.pdf")
parsing
$outFilename = $outpath+$result.Headers.'Content-Disposition'.Split("=")[1].Split('\\""')[1] #shaky parsing - might require improvement in time
Writing the file (this is mostly pdf's for me
[System.IO.File]::WriteAllBytes($outFilename, $result.content)
powershell.exe Invoke-WebRequest -Uri serverIP/file.exe -OutFile C:\Users\file.exe