Cannot unzip zip file downloaded with Powershell - powershell

I have a zip file hosted at a URL. I'm writing a Powershell script to get that zip file using Invoke-RestMethod and save it to disk. When I run the Powershell script, the zip file is downloaded and saved to disk, however, I cannot open or extract it. Does anyone know what I'm doing wrong? Here is my code that handles the zip download/save. Here is my code (I've changed the URL for the zip for security reasons. If I type in the actual URL into a browser it will be downloaded by the browser after which I can locate it on disk and extract it).
$username = "uname"
$password = "pword"
$boundary = [guid]::NewGuid().ToString().Replace("-", "").Substring(0, 16)
$enc = [System.Text.Encoding]::GetEncoding("utf-8")
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
$headers = #{}
$headers.Add("Host", "api.bitbucket.org")
$headers.Add("Authorization", "Basic $base64AuthInfo")
$repoResult
$repoResult = Invoke-WebRequest -Uri "https://bitbucket.org/teamAccount/repoName/get/master.zip" -Headers $headers -Method GET -Verbose -OutFile .\output.zip
}
EDIT/UPDATE:
Thanks to TesselatedHeckler's comment, I've discovered that the downloaded zip is actually an HTML login page. I also realized that I didn't have my headers (which contains the login info) in Invoke-RestMethod line. I've updated the post and my script. Now, whenever I run my script, I get a (500) Internal Server Error.

Related

powershell download file from delayed Link

I'm trying to download a msi Package from a Page which does a redirection in the background which downloads the file I want.
This is the Page I want to download the msi from:
https://remotedesktopmanager.com/de/home/thankyou/rdmmsi
I tried several PowerShell Scripts, but none of them extracted the right download URL or downloaded the file.
With Invoke-Webrequest -Outfile C:xxx only the HTML is saved.
the page uses javascript with a timeout to redirect to the current setup file. that's why you cannot use Invoke-WebRequest as this uses the Internet Explorer engine under the hood which also performs this javascript window.location redirect (resulting in opening the url in the default browser).
To only get the raw HTML you have to use Invoke-RestMethod
$website = Invoke-RestMethod -Uri 'https://remotedesktopmanager.com/de/home/thankyou/rdmmsi'
The full website is now stored in the variable $website without interpreting the javascript.
To find the line with the string window.location i use Select-String which requires a file to be parsed. Thus the content of the variable is first stored in the file which is then parsed.
$tmpFilePath = 'C:\tmp\t.txt'
Out-File -FilePath $tmpFilePath -InputObject $website
$urlRedirectLine = Select-String -Path $tmpFilePath -SimpleMatch "window.location"
Remove-Item -Path $tmpFilePath
The new variable $urlRedirectLine (which content now is C:\tmp\t.txt:999: setTimeout(function () { window.location = 'https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2021.1.25.0.msi'; }, 4500);) contains the string we are looking for.
To extract that url i convert to variable to string and then use SubString() to extract the url itself. For this i look for the first ' and last ' in that variable.
$urlString = $urlRedirectLine.ToString()
$url = $urlString.Substring($urlString.IndexOf("'")+1,$urlString.LastIndexOf("'")-$urlString.IndexOf("'")-1)
Resulting in $url having the url https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2021.1.25.0.msi
To download the file you can again use Invoke-RestMethod
Invoke-RestMethod -Uri $url -OutFile 'C:\tmp\rdm.msi'
I was facing similar error while trying to download httpd.
Following worked for me (-UserAgent "NativeHost")
$ENV:HTTPD_DOWNLOAD_URL = "https://www.apachelounge.com/download/VS16/binaries/httpd-2.4.52-win64-VS16.zip"
$ENV:HTTPD_DOWNLOAD_ZIP = "httpd.zip"
Invoke-WebRequest -Uri $ENV:HTTPD_DOWNLOAD_URL -OutFile $ENV:HTTPD_DOWNLOAD_ZIP -UserAgent "NativeHost";
Reference
https://social.technet.microsoft.com/Forums/en-US/23fccc30-5f84-4a84-8160-c6e95102b11c/powershell-invokewebrequest-sourceforge-urlsredirectiondynamic-content?forum=winserverpowershell
The Link you give is of main download page which redirects after few second to the download.
The downloading link is "https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2021.1.25.0.msi"
Use the following command to download that MSI.
$url = "https://cdn.devolutions.net/download/Setup.RemoteDesktopManager.2021.1.25.0.msi"
$dest = "C:\Setup.RemoteDesktopManager.2021.1.25.0.msi"
Start-BitsTransfer -Source $url -Destination $dest
You can also use Invoke-WebRequest using the above variable
Invoke-WebRequest -Uri $url -OutFile $dest
Thank You

PowerShell - downloads TXT file with WRONG content using the REST API

I have a file which I download using the REST API.
Just to emphasize that I tried with both commands: Invoke-RestMethod and Invoke-WebRequest
$uri = "https://bitbucket.org.dev/projects/TEST/repos/fa/browse/Packages/ATS.txt"
$API_KEY="ZTU2MT"
Invoke-WebRequest -Headers #{Authorization=$("Basic {0}" -f $API_KEY)} -Uri $uri -OutFile ATS.txt
If I access that URI in browser file or download it manually file can be viewd without any issue in clear way.
This is the content of the file (its begining)
#
# exported extension module.
# source ""
# timestamp (utc) "2020-03-30 12:06:23.3"
# ***** DO NOT EDIT! *****
But download file looks completely different (like it is in HTML format)
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta http-equiv="X-..."
Also I have some zip file which needs to be downloaded as well, but with it also I am getting invalid file which cannot be extracted (opened)
As written in MSDN, Invoke-WebRequest returns an object of type BasicHtmlWebResponseObject.
You need to select the Content property to get the text you are looking for.
Invoke-WebRequest -Headers #{Authorization=$("Basic {0}" -f $API_KEY)} -Uri $uri | Select-Object -Expand Content | Out-File -FilePath c:\somefile.txt
Update:
Check this post to find more info about downloading files from a private repository in BitBucket.
https://stackoverflow.com/a/50222671/13440610

Upload Files to Azure Web App using Kudu Rest API - 409 Error

I'm trying to upload some files to my webapp using Kudu to the below url:
https://($websitename.scm.azurewebsites.net/api/vfs/site/wwwroot/ using powershell
I have obtained the publishing username and password and can authenticate fine
However when i try to upload the files i'm getting the below error using the below code:
function Upload-FileToWebApp($kuduHeader,$KuduURL,$files)
{
$kuduURL = https://$websitename.scm.azurewebsites.net/api/vfs/site/wwwroot/
$result = Invoke-RestMethod -Uri $kuduUrl `
-Headers #{Authorization=$kuduheader;"If-Match"="*"} `
-Method PUT `
-InFile $files `
-ContentType "multipart/form-data"
Invoke-RestMethod : {"Message":"The resource represents a directory which can not be updated."}
I have tried to access this URL using the ARC chrome addin and this brings back the same error '409 conflict Message": "The resource represents a directory which can not be updated."
Get seems to work fine
Thanks in Advance!
The problem is that you are doing a PUT on a directory, which has the semantic of creating the directory, when what you're trying to do is upload a file.
You need to change https://$websitename.scm.azurewebsites.net/api/vfs/site/wwwroot/ to https://$websitename.scm.azurewebsites.net/api/vfs/site/wwwroot/MyFile.txt.
And note that the vfs API can only upload one file at a time. If you want to upload multiple, you can use the zip API. See https://github.com/projectkudu/kudu/wiki/REST-API#zip for details.

Invoke-WebRequest pass url as script argument (PowerShell)

I am trying to download file from the web using following command
Invoke-WebRequest $url -OutFile $filePath -Headers $Headers
I have argument, which contains this url and it is passed as parameter
[string]$artifactHttpAddress = $args[2]
Currently its value is
http://10.45.48.26/httpAuth/repository/downloadAll/TeamCityTest_Build/529:id/artifacts.zip
So, when I try to invoke WebRequest with following command
Invoke-WebRequest $artifactHttpAddress -OutFile c:/test.zip -Headers $Headers
it is downloading empty zip file .
but when I try to assign this url to the variable and invoke web request
$url = "http://10.45.48.26/httpAuth/repository/downloadAll/TeamCityTest_Build/529:id/artifacts.zip"
Invoke-WebRequest $url -OutFile c:/test.zip -Headers $Headers
It is working correctly, downloads zip file, which have some content in it.
I tried following script
Write-Host([string]$url -eq [string]$artifactHttpAddress)
Write-Host([string]$url)
Write-Host([string]$artifactHttpAddress)
It outputs
False
http://10.45.48.26/httpAuth/repository/downloadAll/TeamCityTest_Build/528:id/artifacts.zip
http://10.45.48.26/httpAuth/repository/downloadAll/TeamCityTest_Build/531:id/artifacts.zip
What is happening and why?
p.s. this script is inside ScriptBlock
It looks to me, based on your output, that $url and $artifactHttpAddress are not the same value. Does the ZIP file exist at the URL with 531 in it?

PowerShell Invoke-WebRequest, how to automatically use original file name?

How can I use Invoke-WebRequest to download a file but automatically make the file name the same as if I downloaded via browser? I haven't found a way to make -OutFile work without manually specifying the file name. I'm fine with this involving a few other lines of code.
A good solution will:
Work even if the file name isn't in the request URL. For example, the URL to download the Visual Studio x64 Remote Debugging Tools is http://go.microsoft.com/fwlink/?LinkId=393217 but it downloads the file rtools_setup_x64.exe.
Not save the whole file to memory before writing to disk, unless that's what Invoke-WebRequest already does even with the -OutFile parameter (?)
Thanks!
For the example given you're going to need to get the redirected URL, which includes the file name to be downloaded. You can use the following function to do so:
Function Get-RedirectedUrl {
Param (
[Parameter(Mandatory=$true)]
[String]$URL
)
$request = [System.Net.WebRequest]::Create($url)
$request.AllowAutoRedirect=$false
$response=$request.GetResponse()
If ($response.StatusCode -eq "Found")
{
$response.GetResponseHeader("Location")
}
}
Then it's a matter of parsing the file name from the end of the responding URL (GetFileName from System.IO.Path will do that):
$FileName = [System.IO.Path]::GetFileName((Get-RedirectedUrl "http://go.microsoft.com/fwlink/?LinkId=393217"))
That will leave $FileName = rtools_setup_x64.exe and you should be able to download your file from there.
Try this method (may not always work because the file name may not be in the response header)
call Invoke-WebRequest to get the result. then you can inspect the result to look at what is in the headers.
get the file name from the response header (this could be in Headers.Location, or some other place. When I run my query for a url that i was troubleshooting, I found it in the Headers["Content-Disposition"] and it looks like inline; filename="zzzz.docx"
Create a new file based on the name and write the content to this file
Here is code sampe:
$result = Invoke-WebRequest -Method GET -Uri $url -Headers $headers
$contentDisposition = $result.Headers.'Content-Disposition'
$fileName = $contentDisposition.Split("=")[1].Replace("`"","")
$path = Join-Path $yourfoldername $fileName
$file = [System.IO.FileStream]::new($path, [System.IO.FileMode]::Create)
$file.write($result.Content, 0, $result.RawContentLength)
$file.close()
Thanks to Ryan I have a semi-usable function:
Function Get-Url {
param ( [parameter(position=0)]$uri )
invoke-webrequest -uri "$uri" -outfile $(split-path -path "$uri" -leaf)
}
A graphic file and xml file I have been able to download. When I try to download this webpage and open it with Edge it will work at times.
Essence of my code - this works.. using PS 5.1
I have commented out the normal -Outfile statement because that would imply I knew the filename in advance.
I have put this together based upon a number of sources, there are lots of ways of parsing the Content-Disposition headers so use whatever works for you.
$outpath = "C:\\temp\\"
The call:
$result = Invoke-WebRequest -method GET -Uri $resourceUrl -Headers
$resourceHeaders -Verbose #-OutFile $($outPath+"$nodeId.pdf")
parsing
$outFilename = $outpath+$result.Headers.'Content-Disposition'.Split("=")[1].Split('\\""')[1] #shaky parsing - might require improvement in time
Writing the file (this is mostly pdf's for me
[System.IO.File]::WriteAllBytes($outFilename, $result.content)
powershell.exe Invoke-WebRequest -Uri serverIP/file.exe -OutFile C:\Users\file.exe