Invoke-WebRequest doesn't work until I call WebClient.DownloadString - powershell

I'm using powershell v5 to call an internal API using TLS1.2 with a self-signed cert. When I call the api I always get Invoke-WebRequest : The underlying connection was closed: An unexpected error occurred on a send.
E.g.:
PS> [System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
PS> [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
PS> $url = "https://someinternalserver/blah"
PS> $response = Invoke-WebRequest $url
Invoke-WebRequest : The underlying connection was closed: An unexpected error occurred on a send.
At line:1 char:1
(I've looked in the error object but I don't see anything helpful.)
However if I call the same URL using an instance of WebClient, then the call using WebClient AND all subsequent powershell calls works fine:
PS> $webClient = New-Object System.Net.WebClient
PS> $str = $webClient.DownloadString($url)
PS> Write-Host $str
body of request
PS> $response = Invoke-WebRequest $url
PS> Write-Host $response.Content
body or request
I'm not sure what's going on, but I suspect it has something to do with the self-signed cert, or the crypto. Here's what chrome says about the crypto:
I've used powershell to call APIs with self-signed certs before but never had these kind of issues.
Resolution: I'd like to call the API without first using WebClient.
Thanks.

So dug into the error more and found this in an inner exception:
There is no Runspace available to run scripts in this thread.
You can provide one in the DefaultRunspace property of the System.Management.Automation.Runspaces.Runspace type.
The script block you attempted to invoke was: $true
Which led me here:
Powershell 3.0 Invoke-WebRequest HTTPS Fails on All Requests
Which led me here:
https://stackoverflow.com/a/15841856/6311875
Using that code instead of the {$true} did the trick.
So, this further reinforces the idea that all questions are already answered on SO, you just have to look hard enough.

Related

PowerShell Invoke-RestMethod Could not create TLS/SSL secure channel

I'm calling Invoke-RestMethod in a PowerShell script to upload a zip archive to an Artifactory repository.
I tested the script from my local machine PowerShell ISE; upload complete without issue as expected. However, when I execute the script on the intended target machine, I'm seeing an error returned:
The request was aborted: Could not create SSL/TLS secure channel.
I've read numerous online posts related to this error; consensus appears to be to add the following line immediately before the Invoke-RestMethod call in the script:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12 -bor [Net.SecurityProtocolType]::Ssl3
I added this line but the issue persists.
Execution of the following PowerShell:
[Net.ServicePointManager]::SecurityProtocol
returns:
Ssl3, Tls, Tls11, Tls12
suggesting that all available protocols are enabled.
My script snippet now looks like this:
$headers = #{"X-JFrog-Art-Api" = $artifactoryApiKey}
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12 -bor [Net.SecurityProtocolType]::Ssl3
$return=Invoke-RestMethod -Uri $uri -InFile $sourceFile -Method Put -Headers $headers
As stated above, this script executes without issue on my local machine.
.Net 4.6.1 installed on server returning error.
I don't offer an explanation, but perhaps a solution for some. I was having the same exact issue: PowerShell invoke-RestMethod worked locally but not when run from the intended (Windows 2012 R2) server. I receive the same TLS/SSL error. I read a lot about the TLS handshake and other technotes, and tested suggestions like:
Setting the security protocol as noted in the original question
Enabling TLS protocol in the registry
Listing and comparing Cipher Suites.
In the end, I couldn't get PowerShell to work. HOWEVER, the rest call works from the server using curl.exe in command line. If you see this, I recommend stop trying to troubleshoot PowerShell and give curl a try. I found this and this helpful in writing my first curl.exe rest calls.

Powershell Invoke-WebRequest SSL/TLS [duplicate]

I'm trying to execute this powershell command
Invoke-WebRequest -Uri https://apod.nasa.gov/apod/
and I get this error. "Invoke-WebRequest : The request was aborted: Could not create SSL/TLS secure channel." https requests appear to work ("https://google.com") but not this one in question. How can I get this to work or use other powershell command to read the page contents?
try using this one
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest -Uri https://apod.nasa.gov/apod/
In a shameless attempt to steal some votes, SecurityProtocol is an Enum with the [Flags] attribute. So you can do this:
[Net.ServicePointManager]::SecurityProtocol =
[Net.SecurityProtocolType]::Tls12 -bor `
[Net.SecurityProtocolType]::Tls11 -bor `
[Net.SecurityProtocolType]::Tls
Or since this is PowerShell, you can let it parse a string for you:
[Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls"
Then you don't technically need to know the TLS version.
I copied and pasted this from a script I created after reading this answer because I didn't want to cycle through all the available protocols to find one that worked. Of course, you could do that if you wanted to.
Final note - I have the original (minus SO edits) statement in my PowerShell profile so it's in every session I start now. It's not totally foolproof since there are still some sites that just fail but I surely see the message in question much less frequently.
The cause of the error is Powershell by default uses TLS 1.0 to connect to website, but website security requires TLS 1.2. You can change this behavior with running any of the below command to use all protocols. You can also specify single protocol.
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls, [Net.SecurityProtocolType]::Tls11, [Net.SecurityProtocolType]::Tls12, [Net.SecurityProtocolType]::Ssl3
[Net.ServicePointManager]::SecurityProtocol = "Tls, Tls11, Tls12, Ssl3"
After running these commands, try running your command:
Invoke-WebRequest -Uri https://apod.nasa.gov/apod/
then it will work.
If, like me, none of the above quite works, it might be worth also specifically trying a lower TLS version alone. I had tried both of the following, but didn't seem to solve my problem:
[Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls
In the end, it was only when I targetted TLS 1.0 (specifically remove 1.1 and 1.2 in the code) that it worked:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls
The local server (that this was being attempted on) is fine with TLS 1.2, although the remote server (which was previously "confirmed" as fine for TLS 1.2 by a 3rd party) seems not to be.
Hope this helps someone.
It works for me...
if (-not ([System.Management.Automation.PSTypeName]'ServerCertificateValidationCallback').Type)
{
$certCallback = #"
using System;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
public class ServerCertificateValidationCallback
{
public static void Ignore()
{
if(ServicePointManager.ServerCertificateValidationCallback ==null)
{
ServicePointManager.ServerCertificateValidationCallback +=
delegate
(
Object obj,
X509Certificate certificate,
X509Chain chain,
SslPolicyErrors errors
)
{
return true;
};
}
}
}
"#
Add-Type $certCallback
}
[ServerCertificateValidationCallback]::Ignore()
Invoke-WebRequest -Uri https://apod.nasa.gov/apod/
Make sure you switch the SHELL first:
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
RUN [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
RUN Invoke-WebRequest -UseBasicParsing -Uri 'https://github.com/git-for-windows/git/releases/download/v2.25.1.windows.1/Git-2.25.1-64-bit.exe' -OutFile 'outfile.exe'
I haven't figure out the reason but reinstalling the .pfx certificate(both in current user and local machine) works for me.

access sharepoint REST api using powershell with windows authentication

I have a sharepoint 2013 server, which I can log into using Windows Authentication via a web-browser. When I have logged on using my browser, if I then - using that same browser - browse to http://mysharepointserver/_api/web I get some useful XML.
However, I want to write a powershell script that uses this XML. To that end, following the suggestions from this page:
How to make an authenticated web request in Powershell?
I wrote the following code:
$PWord = ConvertTo-SecureString –String "MyAwesomePassword" –AsPlainText -Force
$creds = New-Object System.Net.NetworkCredential("MyUserName",$PWord,"TheDomain")
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = $creds
$url = "http://mysharepointserver/_api/web"
$output = $webclient.DownloadString($url)
echo $output
However, when I run this code, I get the error message:
Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (403) Forbidden."
Even though I can access the same URL using a web-browser, if I type in the (same) credentials using the Windows login dialog that pops up for authentication purposes.
Does anyone know if it is possible to do this using Powershell?
thanks heaps all
Why aren't you using the default CMDlets that come with Powershell (unless using PS2 or lower)?
Invoke-Webrequest and Invoke-Restmethod both support the -credential switch which can be used to authenticate against the webserver.
You can also use Get-credential to store your credentials (though you can't get yhem from a file or string that way, at least not the password)
Script (in pseudo code and since i'm on my phone no code tags) would look like this :
Invoke-Restmethod -uri 'http://bla/api' -credential (get-credential)

Powershell and NITRO API for Citrix NetScaler error on GET method

I am using a PowerShell module provided by Citrix to invoke the Nitro REST API. Calling the function I can successfully add and remove load balanced services from the load. However when I try to do a GET method to get the status of a service I get the error:
Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send.
I have tried running Invoke-RestMethod without using the module but get the same error
Invoke-RestMethod -WebSession $myNSSession.WebSession -Method GET -Uri https://<NetScaler IP/nitro/v1/config/service/<Service Name>
When googling this error everything seems to point to certificate issues. I had this initially even on POST method until i added the below to my script
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
So since this works for doing POST i cant see why it wouldn't for a GET!!
another weird thing is, if I put the URL directly into the browser then enter my credentials i get a response in raw text! so it looks like this is an issue with the way i am calling it in PowerShell rather than the NetScaler or the NITRO API!
Someone please help as this is driving me crazy!!
Admitedly i am new to Invoke-RestMethod commands, but try this:
$creds = Get-Credential
$service = Invoke-RestMethod -Uri https://<NetScaler IP/nitro/v1/config/service/<Service Name> -Credential $creds
What you will get is something similar to this:
*errorcode* *message* *serverity* *service*
* 0 Done NONE {#{name=<service name; n..
then type $service.service and you will see more information. whatever attributes are availible will be listed. then just follow the pattern:
$service.service.
I had the same problem with Nitro API (specifically v10.5), and found that setting certificate policies, TLS versions and trust settings had no effect. POST works, GET fails.
The solution for me was to not use the cmdlets and instead drop back to a native .Net method. Below I am still using HTTPS with an internal certificate, hence still setting the callback.
$NSProtocol = "https://"
$NSHostname = "netscaler"
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$WebRequest = [System.Net.WebRequest]::Create("$NsProtocol$NsHostname/nitro/v1/config/hanode")
$WebRequest.Method = "GET"
$WebRequest.ContentType = "application/json; charset=utf-8";
$WebRequest.Headers.Add("AUTHORIZATION","Basic $([System.Convert]::ToBase64String([System.Text.Encoding]::GetEncoding("ISO-8859-1").GetBytes($nsuser+":"+$nspass)))")
$Response = $WebRequest.GetResponse()
$ReadStream = New-Object System.IO.StreamReader $Response.GetResponseStream()
$HaState = ConvertFrom-Json $ReadStream.ReadToEnd()
Hope that helps.

http requests with powershell

I am looking to make http requests to web pages with powershell, is this possible and if so, how may I achieve this?
Can I make requests to https pages? I am able to make http requests with a bat file but not https, was hoping I could https page requests with powershell.
You can use the usual WebRequest and HttpWebRequest classes provided by the .NET framework.
$request = [System.Net.WebRequest]::Create('http://example.com')
# do something with $request
It's no different from using the same classes and APIs from C#, except for the syntactic differences to PowerShell.
PowerShell v3 also brings Invoke-WebRequest and a few others.
Try this:
(New-Object System.Net.WebClient).DownloadString("http://stackoverflow.com")
WebClient.DownloadString Method (String)
or in PowerShell 3.0,
(Invoke-WebRequest http://stackoverflow.com).content
Invoke-WebRequest
Depending on what you are doing, you can also use System.Net.WebClient, which is a simplified abstraction of HttpWebRequest
$client = new-object system.net.webclient
Look here for difference: What difference is there between WebClient and HTTPWebRequest classes in .NET?
PS: With Powershell v3.0, you have Invoke-WebRequest and Invoke-RestMethod cmdlets which can be used for similar purposes
If all else fails, use Curl from http://curl.haxx.se . You can set everything, including certificate handling, POSTs, etc. Not subtle, but it works and handles all of the odder cases; e.g. you can set the --insecure flag to ignore certificate name issues, expiration, or test status.
You can create HTTP, HTTPS, FTP and FILE requests using Invoke-WebRequest cmdlet. This is pretty easy and gives many options to play around.
Example: To make simple http/https requests to google.com
Invoke-WebRequest -Uri "http://google.com"
More references can be found MSDN
This code works with both ASCII & binary files over https in powershell:
# Add the necessary .NET assembly
Add-Type -AssemblyName System.Net.Http
# Create the HttpClient object
$client = New-Object -TypeName System.Net.Http.Httpclient
# Get the web content.
$task = $client.GetByteArrayAsync("https://stackoverflow.com/questions/7715695/http-requests-with-powershell")
# Wait for the async call to finish
$task.wait();
# Write to file
[io.file]::WriteAllBytes('test.html',$task.result)
Tested on Powershell 5.1.17134.1, Win 10
Try this PowerShell module: https://github.com/toolkitx/simple-request
Install by Install-Module -Name SimpleRequest
Then you can send requests like
$Data = #{
"TokenUrl" = 111
"ClientSecret" = "222"
"ClientId" = "333"
"AuthResource" = "444"
"Username" = "User1"
"Password" = "Password"
"Id" = 99
"Price" = 0.99
"Value" = "Content"
}
$Sample = '
POST https://httpbin.org/post?id={{Id}}
Content-Type: application/json
Authorization: Bearer {{QIBToken}}
{
"id": {{Id}},
"value": "{{Value}}"
}'
$Response = Invoke-SimpleRequest -Syntax $Sample -Context $Data
Please refer to GitHub for detail introductions
This method downloads the content:
# PowerShell 2 version
$WebRequest=New-Object System.Net.WebClient
$WebRequest.UseDefaultCredentials=$true
#$WebRequest.Credentials=(Get-Credential)
$Data=$WebRequest.DownloadData("http://<url>")
[System.IO.File]::WriteAllBytes("<full path of file>",$Data)
# PowerShell 5 version
Invoke-WebRequest -Uri "http://<url>" -OutFile "<full path of file>" -UseDefaultCredentials -ContentType