Using PowerShell 4.0 and Invoke-RestMethod cmdlet. I'm having trouble with the -OutFile and -PassThru options. Whenever I add the -PassThru option, my -OutFile is created but the contents are Empty!
According to the Invoke-RestMethod Documentation, both an output file and pipeline object should be available when these options are used together. "-OutFile Saves the response body in the specified output file. [...] To send the results to a file and to the pipeline, use the Passthru parameter."
Here's a test to repeat the problem I'm having. Here I'm calling a rest api attempting to BOTH save response to file AND deserialize into a powershell object.
"POWERSHELL VERSION $($host.Version.ToString())"
$date = Invoke-RestMethod "http://date.jsontest.com" -OutFile "OutFile.txt" -PassThru
Get-Content "OutFile.txt"
# FILE IS EMPTY!!! PASSTHRU SEEMS TO RESULT IN EMPTY FILE
$date
# powershell object has the date received from api
Here are two tests to verify the normal functionality of Invoke-RestMethod WITHOUT the PassThru option
# ... Test # 1, call rest api and deserialize into powershell object
$date = Invoke-RestMethod "http://date.jsontest.com"
$date
# Output shows the date retrieved from sample restful service
# ... Test # 2, call rest api and save response body directly to a file
Invoke-RestMethod "http://date.jsontest.com" -OutFile "OutFile.txt"
Get-Content "OutFile.txt"
# Output shows contents of rest api response body (json text)
I think these tests should help others see the trouble I'm having. My question is whether there is something I'm missing to make this work, or whether this may be a bug with the cmdlet? I've Googled a bit for solution and no obvious reports of this issue. I'm wanting to use -OutFile as part of a workaround for another Invoke-RestMethod issue related to content encoding as described at Bug? Invoke-RestMethod and UTF-8 data. The -PassThru option is helpful for me to look at the returned data and terminate iteration on a multi-request (paged) odata result set.
I believe the -PassThru switch redirects all output to the console only, and I think that is why your file is empty. However, since you have it a variable you could add one more line like so. . .
Write-Output -InputObject $date | Out-File -FilePath "OutFile.txt"
Related
I am trying to perform CI/CD using Perfecto and hence I am trying to upload a file to perfecto when my Bamboo build is finished.
I was trying with the following cURL command when we have a Linux server.
curl -X POST --upload-file test.apk 'https://****.perfectomobile.com/services/repositories/media/PRIVATE:test.apk?operation=upload&user=<email>&password=<password>&overwrite=true'
Now our server is changed to Windows and hence I want a powershell script which I can use as an Inline Scripts in Bamboo.
Can you please tell me what is an equivalent script in Powershell for windows.
Many thanks in advance.
# Gather your information.
$email = "myEmail#website.com";
$password = "powershellR0cks!";
$subDomain = "****";
$url = "https://$subDomain.perfectomobile.com/services/repositories/media/PRIVATE:test.apk?operation=upload&user=$email&password=$password&overwrite=true";
$filePath = ".\test.apk";
# Make the request.
$response = Invoke-WebRequest -Uri $URL -Method Post -InFile $filePath -ContentType "application/octet-stream";
# Check for success.
if (-not ($response.StatusCode -eq 200)) {
throw "There was an error uploading the APK manifest.";
}
You may want to check the value of -ContentType, but I think that's correct. You don't necessarily need to include the scheme (HTTPS) if you don't want to, and semicolons in PowerShell are optional, but you can include them if you want.
The $response variable is an HtmlWebResponseObject that has the content of the response, the status code, and a bunch of other useful info. You can check out the available properties and methods on the object by running $response | Get-Member.
Finally, the Invoke-WebRequest cmdlet also has other parameters that may be useful to you, such as -Credential, -Headers, and more.
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-5.1
As a side-note, if you run Get-Alias -Name "curl", you can see that anytime you use curl in PowerShell, you're really just calling Invoke-WebRequest. You can use the curl alias if you want, but it's generally not a good idea to use aliases in automation since they can be modified or deleted.
I have a URL to a CSV file which, in a browser, I can download and open without issue.
I'm trying to download this file using PowerShell without success. I tried using Invoke-WebRequest, Start-BitsTransfer and using a webrequest object but no luck there.
Invoke-WebRequest comes with a parameter to store its result in a file: -OutFile
Invoke-WebRequest $myDownloadUrl -OutFile c:\file.ext
If you need authorization before you can send a request like this:
Invoke-WebRequest $myAuthUrl /* whatever is neccesary to login */ -SessionVariable MySession
Invoke-WebRequest $myDownloadUrl -WebSession $MySession
To determine the layout of the form where the login happens, you can use Invoke-WebRequests return object. It'll collect information about forms and fields on the HTML (might be Windows only). Mileage of logging in may vary with things like Two-Factor-Auth active or not. Probably you can create some secret link to your file which does not need Auth or possibly google allows you to create a private access token of some sort, which can be send aus Authorization-Header alongside your request.
TLDR answers*:
Method 1, by default synchronous**
Invoke-WebRequest $url -OutFile $path_to_file
(if you get error "...Could not create SSL/TLS secure channel." see Powershell Invoke-WebRequest Fails with SSL/TLS Secure Channel)
Method 2, by default synchronous**
(New-Object System.Net.WebClient).DownloadFile($url, $path_to_file)
Method 3, asynchronous and may be much slower than the other two but is very gentle on bandwidth usage (it uses the BITS service).
Import-Module BitsTransfer
Start-BitsTransfer -Source $url -Destination $path_to_file
Notes:
*: This answer is for those that google for "how to download a file with PowerShell".
**: Read the help pages if you want asynchronous downloading
For a while now I've been using a PS script to download PowerBI bi-monthly and using the BITS, it's been pretty solid and now so much stronger now since I removed the -Asynchronous at the end of the Start-BitsTransfer
$url = "https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup.exe"
$output = "%RandomPath%\PowerBI Pro\PBIDesktopSetup.exe"
$start_time = Get-Date
Import-Module BitsTransfer
Start-BitsTransfer -Source $url -Destination $output
#Commented out below because it kept creating "Tmp files"
#Start-BitsTransfer -Source $url -Destination $output -Asynchronous
I assembled a Powershell script that is designed to grab other scripts that are hosted on Azure blobs, and execute them.
The relevant code blocks:
Obtaining the script:
$resp = (Invoke-WebRequest -Uri $scriptUri -Method GET -ContentType "application/octet-stream;charset=utf-8")
$migrationScript = [system.Text.Encoding]::UTF8.GetString($resp.RawContentStream.ToArray());
$tempPath = Get-ScriptDirectory
$fileLocation = CreateTempFile $tempPath "migrationScript.ps1" $migrationScript
Creating the file:
$newFile = "$tempFolder"+"\"+"$fileName"
Write-Host "Creating temporary file $newFile"
[System.IO.File]::WriteAllText($newFile, $fileContents)
And then I invoke the downloaded file with
Invoke-Expression "& `"$fileLocation`" $migrationArgs"
This is working well, for what I need. However, the Invoke-Expression is not correctly reading the encoding of the file. It opens correctly in Notepad or Notepad++, but not in ISE (where I am executing the script right now).
Is there a way I can ensure the script is read correctly? It is necessary to support UTF8, as there is a possibility that the scripts will need to perform operations such as setting an AppSetting to a value that contains special characters.
EDIT: Behaviour is the same on "vanilla" non-ISE Powershell invocation.
As per #lit and #PetSerAI, the BOM is required for Powershell to work correctly.
My first attempt had not been successful, so I switched back to non-BOM, but, with the following steps, it worked:
Perform the Invoke-WebRequest with -ContentType "application/octet-stream;charset=utf-8"
Grab the Raw content (you will see it in Powershell as a series of numbers, which I assume are the ascii codes?) and convert its bytes with [system.Text.Encoding]::UTF8.GetString($resp.RawContentStream.ToArray()); to an array containing the characters you want.
When saving the file via .NET's WriteAllText, ensure you use UTF8,
[System.IO.File]::WriteAllText($newFile, $fileContents, [System.Text.Encoding]::UTF8). In this case, UTF8 is understood to be UTF8 with a byte order mark, and is what Powershell needs.
How can I use Invoke-WebRequest to download a file but automatically make the file name the same as if I downloaded via browser? I haven't found a way to make -OutFile work without manually specifying the file name. I'm fine with this involving a few other lines of code.
A good solution will:
Work even if the file name isn't in the request URL. For example, the URL to download the Visual Studio x64 Remote Debugging Tools is http://go.microsoft.com/fwlink/?LinkId=393217 but it downloads the file rtools_setup_x64.exe.
Not save the whole file to memory before writing to disk, unless that's what Invoke-WebRequest already does even with the -OutFile parameter (?)
Thanks!
For the example given you're going to need to get the redirected URL, which includes the file name to be downloaded. You can use the following function to do so:
Function Get-RedirectedUrl {
Param (
[Parameter(Mandatory=$true)]
[String]$URL
)
$request = [System.Net.WebRequest]::Create($url)
$request.AllowAutoRedirect=$false
$response=$request.GetResponse()
If ($response.StatusCode -eq "Found")
{
$response.GetResponseHeader("Location")
}
}
Then it's a matter of parsing the file name from the end of the responding URL (GetFileName from System.IO.Path will do that):
$FileName = [System.IO.Path]::GetFileName((Get-RedirectedUrl "http://go.microsoft.com/fwlink/?LinkId=393217"))
That will leave $FileName = rtools_setup_x64.exe and you should be able to download your file from there.
Try this method (may not always work because the file name may not be in the response header)
call Invoke-WebRequest to get the result. then you can inspect the result to look at what is in the headers.
get the file name from the response header (this could be in Headers.Location, or some other place. When I run my query for a url that i was troubleshooting, I found it in the Headers["Content-Disposition"] and it looks like inline; filename="zzzz.docx"
Create a new file based on the name and write the content to this file
Here is code sampe:
$result = Invoke-WebRequest -Method GET -Uri $url -Headers $headers
$contentDisposition = $result.Headers.'Content-Disposition'
$fileName = $contentDisposition.Split("=")[1].Replace("`"","")
$path = Join-Path $yourfoldername $fileName
$file = [System.IO.FileStream]::new($path, [System.IO.FileMode]::Create)
$file.write($result.Content, 0, $result.RawContentLength)
$file.close()
Thanks to Ryan I have a semi-usable function:
Function Get-Url {
param ( [parameter(position=0)]$uri )
invoke-webrequest -uri "$uri" -outfile $(split-path -path "$uri" -leaf)
}
A graphic file and xml file I have been able to download. When I try to download this webpage and open it with Edge it will work at times.
Essence of my code - this works.. using PS 5.1
I have commented out the normal -Outfile statement because that would imply I knew the filename in advance.
I have put this together based upon a number of sources, there are lots of ways of parsing the Content-Disposition headers so use whatever works for you.
$outpath = "C:\\temp\\"
The call:
$result = Invoke-WebRequest -method GET -Uri $resourceUrl -Headers
$resourceHeaders -Verbose #-OutFile $($outPath+"$nodeId.pdf")
parsing
$outFilename = $outpath+$result.Headers.'Content-Disposition'.Split("=")[1].Split('\\""')[1] #shaky parsing - might require improvement in time
Writing the file (this is mostly pdf's for me
[System.IO.File]::WriteAllBytes($outFilename, $result.content)
powershell.exe Invoke-WebRequest -Uri serverIP/file.exe -OutFile C:\Users\file.exe
I just want to call an URL with the tasks scheduler on Windows Server 2008 and a Powershell Script.
So I developed the following script :
$url = "http://example.com"
$log_file = "${Env:USERPROFILE}\Desktop\Planification.log"
$date = get-date -UFormat "%d/%m/%Y %R"
"$date [INFO] Exécution de $url" >> $log_file
$request = [System.Net.WebRequest]::Create($url)
$response = $request.GetResponse()
$response.Close()
The script works without any issue when I execute it from the Powershell ISE, the Powershell console or with the command :
powershell PATH_TO_MY_SCRIPT.ps1
But it doesn't works when I execute it from the Scheduler tasks, it returns the code 0xFFFD0000. I found this : http://www.briantist.com/errors/scheduled-task-powershell-0xfffd0000/
So it can be a permission problem, so I tried many others profiles and option (including "Execute with all permissions") to test, without success.
I execute also another Powershell script which just mount a network drive and copy two files, and I don't have any issue with this script. So I think that the issue come from the using of the .NET object to call my URL.
I saw that I may have to include modules in my script before any other commands, but I don't know exactly what I have to do. (I don't know Powershell, I just try to use it for resolving my problems).
Thank you for you help.
i used the following in a scheduled task and it works as expected :
$url="http://server/uri"
(New-Object System.Net.WebClient).DownloadString("$url");
following code should do what you need.
$url = "http://www.google.com"
PowerShell Invoke-WebRequest -Uri $url -Method GET
You'll want to put that log file out somewhere in a shared directory. Scheduled tasks may or may not have access to $env:USERPROFILE. Also, use Start-Transcript to have PowerShell write your script's output to a log file (except for STDOUT, you'll need to pipe the output from executables to Write-Host, e.g. hostname | Write-Host).
$url = "http://example.com"
$log_file = "C:\PlanificationLogs\Planification.log"
Start-Transcript -Path $log_file
Get-Date -UFormat "%d/%m/%Y %R"
Write-Host "$date [INFO] Exécution de $url"
# PowerShell 3
Invoke-WebRequest -Uri $url
# PowerShell 2
$request = [System.Net.WebRequest]::Create($url)
$response = $request.GetResponse()
$response.Close()