I am trying to access Foreman API in Powershell version 5.1 using Invoke-RestMethod
I get following error:
Invoke-RestMethod : The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
I tried following command before the this command but it didn't work.
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
I tried with Tls, Tls11 also but it didn't work.
Are you connecting to an untrusted SSL certificate by any chance?
You probably need to ignore trust errors as well as set the correct TLS version to use.
Add-Type #"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"#
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
Related
I am having some trouble converting a Powershell script from PS5 to PS7.
My goal is to get information on a remote server cert.
This IP can be anything with a cert:
$url = "https://192.168.1.1"
[Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
$req = [Net.HttpWebRequest]::Create($url)
$req.GetResponse()
[datetime]$expiration =
[System.DateTime]::Parse($req.ServicePoint.Certificate.GetExpirationDateString())
Write-Output $expiration
When run in Powershell 5, this script completes successfully with the output below:
IsMutuallyAuthenticated : False
Cookies : {}
Headers : {Accept-Ranges, Content-Length, Content-Type, Date...}
SupportsHeaders : True
ContentLength : 98540
ContentEncoding :
ContentType : text/html
CharacterSet : ISO-8859-1
Server : Microsoft-IIS/10.0
...
Date : 7/7/2020 12:00:00 AM
...
This script errors on $req.GetResponse when run in Powershell 7.
MethodInvocationException: Exception calling "GetResponse" with "0" argument(s): "The SSL connection could not be established, see inner exception. There is no Runspace available to run scripts in this thread. You can provide one in the DefaultRunspace property of the System.Management.Automation.Runspaces.Runspace type. The script block you attempted to invoke was: $true "
Any ideas?
Recently I encounter some problems making the connection to a FTP server but there will be some popup asking for the acceptance on the certificate.
I don't know how to overcome this via PowerShell during invoke method $ftpRequest.GetResponse(). I found some solution regarding overriding the callback method on certificate like this one [System.Net.ServicePointManager]::ServerCertificateValidationCallback
The solution is given on C# & I don't know how to port it to PowerShell yet.
My code is as below
function Create-FtpDirectory {
param(
[Parameter(Mandatory=$true)]
[string]
$sourceuri,
[Parameter(Mandatory=$true)]
[string]
$username,
[Parameter(Mandatory=$true)]
[string]
$password
)
if ($sourceUri -match '\\$|\\\w+$') { throw 'sourceuri should end with a file name' }
$ftprequest = [System.Net.FtpWebRequest]::Create($sourceuri);
Write-Information -MessageData "Create folder to store backup (Get-FolderName -Path $global:backupFolder)"
$ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::MakeDirectory
$ftprequest.UseBinary = $true
$ftprequest.Credentials = New-Object System.Net.NetworkCredential($username,$password)
$ftprequest.EnableSsl = $true
$response = $ftprequest.GetResponse();
Write-Host "Folder created successfully, status $response.StatusDescription"
$response.Close();
}
[UPDATED] While searching for Invoke-RestRequest, I found this solution from Microsoft example
Caution: this is actually accept ANY Certificate
# Next, allow the use of self-signed SSL certificates.
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $True }
More information (thanks to #Nimral) : https://learn.microsoft.com/en-us/dotnet/api/system.net.servicepointmanager.servercertificatevalidationcallback?view=netcore-3.1
It's a bit hacky, but you can use raw C# in PowerShell via Add-Type. Here's an example class I've used to be able to toggle certificate validation in the current PowerShell session.
if (-not ([System.Management.Automation.PSTypeName]'CertValidation').Type)
{
Add-Type #"
using System.Net;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
public class CertValidation
{
static bool IgnoreValidation(object o, X509Certificate c, X509Chain ch, SslPolicyErrors e) {
return true;
}
public static void Ignore() {
ServicePointManager.ServerCertificateValidationCallback = IgnoreValidation;
}
public static void Restore() {
ServicePointManager.ServerCertificateValidationCallback = null;
}
}
"#
}
Then you can use it prior to calling your function like this.
[CertValidation]::Ignore()
And later, restore default cert validation like this.
[CertValidation]::Restore()
Keep in mind though that it's much safer to just fix your service's certificate so that validation actually succeeds. Ignoring certificate validation should be your last resort if you have no control over the environment.
I'm attempting to create a self signed certificate in KeyVault using the "Self" issuer.
$policy = New-AzureKeyVaultCertificatePolicy -SubjectName "CN=$($certificateName)" -IssuerName "Self" -ValidityInMonths 12
$policy.Exportable = $true
Add-AzureKeyVaultCertificate -VaultName $vaultName -Name $certificateName -CertificatePolicy $policy
However, when getting the certificate back it doesn't appear to have a private key.
Creating certificates directly in KeyVault doesn't seem hugely covered online, after digging into the rest API documentation and source code for the powershell cmdlets, I'm stumped.
I'm hoping it's something simple I've missed, as I wish to avoid creating the certificate locally..
If you'd like to retrieve your certificate along with its private key, then you can export it to a PFX file (with an empty password) on your disk via:
$vaultName = "my-vault-name"
$certificateName = "my-cert-name"
$pfxPath = [Environment]::GetFolderPath("Desktop") + "\$certificateName.pfx"
$pfxSecret = Get-AzureKeyVaultSecret -VaultName $vaultName -Name $certificateName
$pfxUnprotectedBytes = [Convert]::FromBase64String($pfxSecret.SecretValueText)
[IO.File]::WriteAllBytes($pfxPath, $pfxUnprotectedBytes)
If you'd like to view just the private key itself in-memory without writing to disk, then try:
$vaultName = "my-vault-name"
$certificateName = "my-cert-name"
$pfxPath = [Environment]::GetFolderPath("Desktop") + "\$certificateName.pfx"
$pfxSecret = Get-AzureKeyVaultSecret -VaultName $vaultName -Name $certificateName
$pfxUnprotectedBytes = [Convert]::FromBase64String($pfxSecret.SecretValueText)
$pfx = New-Object Security.Cryptography.X509Certificates.X509Certificate2
$pfx.Import($pfxUnprotectedBytes, $null, [Security.Cryptography.X509Certificates.X509KeyStorageFlags]::Exportable)
$pfx.PrivateKey.ExportParameters($true)
which will show the private parameters in addition to the exponent and modulus.
If you'd like to protect the PFX file on disk with your own password (as per the "Retrieve pfx file & add password back" instructions in this blog post), then try:
$vaultName = "my-vault-name"
$certificateName = "my-cert-name"
$pfxPath = [Environment]::GetFolderPath("Desktop") + "\$certificateName.pfx"
$password = "my-password"
$pfxSecret = Get-AzureKeyVaultSecret -VaultName $vaultName -Name $certificateName
$pfxUnprotectedBytes = [Convert]::FromBase64String($pfxSecret.SecretValueText)
$pfx = New-Object Security.Cryptography.X509Certificates.X509Certificate2
$pfx.Import($pfxUnprotectedBytes, $null, [Security.Cryptography.X509Certificates.X509KeyStorageFlags]::Exportable)
$pfxProtectedBytes = $pfx.Export([Security.Cryptography.X509Certificates.X509ContentType]::Pkcs12, $password)
[IO.File]::WriteAllBytes($pfxPath, $pfxProtectedBytes)
As mentioned in the REST API docs here and here, Azure Key Vault (AKV) represents a given X.509 certificate via three interrelated resources: an AKV-certificate, an AKV-key, and an AKV-secret. All three will share the same name and the same version - to verify this, examine the Id, KeyId, and SecretId properties in the response from Get-AzureKeyVaultCertificate.
Each of these 3 resources provide a different perspective for viewing a given X.509 cert:
The AKV-certificate provides the public key and cert metadata of the X.509 certificate. It contains the public key's modulus and exponent (n and e), as well as other cert metadata (thumbprint, expiry date, subject name, and so on). In PowerShell, you can obtain this via:
(Get-AzureKeyVaultCertificate -VaultName $vaultName -Name $certificateName).Certificate
The AKV-key provides the private key of the X.509 certificate. It can be useful for performing cryptographic operations such as signing if the corresponding certificate was marked as non-exportable. In PowerShell, you can only obtain the public portion of this private key via:
(Get-AzureKeyVaultKey -VaultName $vaultName -Name $certificateName).Key
The AKV-secret provides a way to export the full X.509 certificate, including its private key (if its policy allows for private key exporting). As demonstrated above, the current base64-encoded certificate can be obtained in PowerShell via:
(Get-AzureKeyVaultSecret -VaultName $vaultName -Name $certificateName).SecretValueText
Following is C# code to retrieve all versions of a certificate, including their private keys, from newest to oldest, given its certificate name and KeyVault connection info. It uses the new Azure.Core, Azure.Identity, and Azure.Security.KeyVault.[Certificates|Secrets] SDK packages.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using Azure.Core;
using Azure.Identity;
using Azure.Security.KeyVault.Certificates;
using Azure.Security.KeyVault.Secrets;
public static class CertTools
{
public static void MyMethod(string tenantId, string clientId, string clientSecret, Uri keyVaultUri)
{
var cred = new ClientSecretCredential(tenantId, clientId, clientSecret); // or any other means of obtaining Azure credential
var certs = GetAllCertificateVersions(keyVaultUri, cred, "MyCert");
}
public static List<X509Certificate2> GetAllCertificateVersions(Uri keyVaultUri, TokenCredential credential,
string certificateName)
{
var certClient = new CertificateClient(keyVaultUri, credential);
var secretClient = new SecretClient(keyVaultUri, credential);
var now = DateTimeOffset.UtcNow;
var certs = new List<X509Certificate2>();
foreach (var cert in certClient.GetPropertiesOfCertificateVersions(certificateName)
.OrderByDescending(x => x.CreatedOn)
// fetch all enabled, non-expired certificates. adjust this predicate if desired.
.Where(x => x.ExpiresOn >= now && (x.Enabled ?? false)))
{
var secret = secretClient.GetSecret(certificateName, cert.Version).Value;
certs.Add(new X509Certificate2(Convert.FromBase64String(secret.Value)));
}
return certs;
}
}
Thanks to #Nandun's answer here for pointing me in the right direction of using the SecretClient instead of CertificateClient, but that post was marked as a duplicate so posting this extended code here.
I have a simple web server running on a wifi chip (esp8266, written in LUA, running on nodeMCU). When i query the website in any browser, i get the response as expected and displayed in the browser (about 45 characters long). When i query it in C# or powershell, i get:
"The underlying connection was closed: The connection was closed
unexpectedly."
I have tried numerous options suggested across many forums but none of them seem to have worked.
Is there any way possible to make a web request the same way IE or Chrome does? I'm not sure what extra steps browsers are doing internally such that they are able to get the response without issue? Why is this an issue in .NET?
My script is below. I am considering just using c# to fire off PhantomJS (headless browser), then using javascript to tell it to open the website, and then pass back the response. Or alternatively, use sockets to open a connection and do it that way, rather than relying on .NET wrappers.
# Set the useUnsafeHeaderParsing property to true
$netAssembly = [Reflection.Assembly]::GetAssembly([System.Net.Configuration.SettingsSection])
$bindingFlags = [Reflection.BindingFlags] "Static,GetProperty,NonPublic"
$settingsType = $netAssembly.GetType("System.Net.Configuration.SettingsSectionInternal")
$instance = $settingsType.InvokeMember("Section", $bindingFlags, $null, $null, #())
if($instance)
{
$bindingFlags = "NonPublic","Instance"
$useUnsafeHeaderParsingField = $settingsType.GetField("useUnsafeHeaderParsing", $bindingFlags)
if($useUnsafeHeaderParsingField)
{
$useUnsafeHeaderParsingField.SetValue($instance, $true)
}
}
# Try setting the certificate policy to a custom child class that always returns true
add-type #"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"#
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
# Try setting other attributes on the ServicePointManager class
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls -bxor [System.Net.SecurityProtocolType]::Ssl3
[System.Net.ServicePointManager]::Expect100Continue = $false;
# Initiate the web request
$r = [System.Net.WebRequest]::Create("http://192.168.1.7/GetStatusAsJson")
# Try long timeouts, with KeepAlive set to false; Also try giving it a user agent string etc.
$r.Timeout = 5000
$r.ReadWriteTimeout = 5000
$r.KeepAlive = $false
$r.Method = "GET"
$r.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36";
$r.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
$resp = $r.GetResponse()
I ended up using MSXML2.XMLHTTP.3.0 with vbscript, and executed the vbscript from powershell.
on error resume next
Dim oXMLHTTP
Dim oStream
Set oXMLHTTP = CreateObject("MSXML2.XMLHTTP.3.0")
oXMLHTTP.Open "GET", WScript.Arguments.Item(2) & "/OpenDoor", False
oXMLHTTP.Send
If oXMLHTTP.Status = 200 Then
Set oStream = CreateObject("ADODB.Stream")
oStream.Open
oStream.Type = 1
oStream.Write oXMLHTTP.responseBody
oStream.SaveToFile WScript.Arguments.Item(0) & "\OpenDoor.html"
oStream.Close
End If
I am trying to work with our Load Balancer via Powershell 3.0 and a REST API. However I am currently getting a failure no matter what I try if it is an https request, whether to our load balancer or to any other https site. I feel like I'm missing something obvious.
Here is the code that fails with https
try
{
#fails
#$location='https://www.bing.com'
#fails
#$location='https://www.google.com'
#fails
#$location='https://www.facebook.com'
#fails
#$location='https://www.ebay.com'
#works
#$location='http://www.bing.com'
#works
#$location='http://www.google.com'
#fails (looks like Facebook does a redirect to https://)
$location='http://www.facebook.com'
#works
#$location='http://www.ebay.com'
$response=''
$response = Invoke-WebRequest -URI $location
$response.StatusCode
$response.Headers
}
catch
{
Write-Host StatusCode $response.StatusCode
Write-Host $_.Exception
}
The error I get is:
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.Management.Automation.PSInvalidOperationException:
There is no Runspace available to run scripts in this thread. You can provide one in the DefaultRunspace property of the System.Management.Automation.Runspaces.Runspa
ce type. The script block you attempted to invoke was: $true
at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult)
at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.GetResponse(WebRequest request)
at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.ProcessRecord()
I was hoping this page and the suggestions towards the bottom including the one from Aaron D.) would make a difference but none of them made a difference.
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
and
function Ignore-SSLCertificates
{
$Provider = New-Object Microsoft.CSharp.CSharpCodeProvider
$Compiler = $Provider.CreateCompiler()
$Params = New-Object System.CodeDom.Compiler.CompilerParameters
$Params.GenerateExecutable = $false
$Params.GenerateInMemory = $true
$Params.IncludeDebugInformation = $false
$Params.ReferencedAssemblies.Add("System.DLL") > $null
$TASource=#'
namespace Local.ToolkitExtensions.Net.CertificatePolicy
{
public class TrustAll : System.Net.ICertificatePolicy
{
public bool CheckValidationResult(System.Net.ServicePoint sp,System.Security.Cryptography.X509Certificates.X509Certificate cert, System.Net.WebRequest req, int problem)
{
return true;
}
}
}
'#
$TAResults=$Provider.CompileAssemblyFromSource($Params,$TASource)
$TAAssembly=$TAResults.CompiledAssembly
## We create an instance of TrustAll and attach it to the ServicePointManager
$TrustAll = $TAAssembly.CreateInstance("Local.ToolkitExtensions.Net.CertificatePolicy.TrustAll")
[System.Net.ServicePointManager]::CertificatePolicy = $TrustAll
}
and
add-type #"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"#
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
I have tried switching to Invoke-RestCommand but to no avail as I get the same response.
It feels like this has to be something environmental because I can't believe the above doesn't work for anyone else, but I've tried it on a workstation and on a server with the same results (doesn't rule out environment completely but I know they were set up differently).
Any thoughts?
This worked perfectly for me. The site defaults to TLS 1.0 and apparently PS doesn't work with that. I used this line:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
My PS scripts (so far all I've tested) have worked perfectly.
The answer is do not do this to solve the SSL issue:
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
If you do this, your first https request will work (it seems), however subsequent ones will not. Additionaly at that point you need to close out of the Powershell ISE, and reopen it and then try again (without that line).
This is alluded to in a sentence here http://social.technet.microsoft.com/Forums/windowsserver/en-US/79958c6e-4763-4bd7-8b23-2c8dc5457131/sample-code-required-for-invokerestmethod-using-https-and-basic-authorisation?forum=winserverpowershell - "And all subsequent runs produce this error :", but it wasn't clear what the solution to reset was.
I too was plagued by this for a really long time. It even affected Visual Studio as VS loaded my $PROFILE into it's domain when running NuGet restore.
Seeing your comment above made me realize that I had a custom callback script because of one of our vendors shipped a product with an invalid CN in it's ssl cert.
Long story short, I replaced my script delegate with a compiled c# object (removing the script runspace from the equation).
(separate code block for C# highlighting)
using System.Net;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
public static class CustomCertificateValidationCallback {
public static void Install()
{
ServicePointManager.ServerCertificateValidationCallback += CustomCertificateValidationCallback.CheckValidationResult;
}
public static bool CheckValidationResult(
object sender,
X509Certificate certificate,
X509Chain chain,
SslPolicyErrors sslPolicyErrors)
{
// please don't do this. do some real validation with explicit exceptions.
return true;
}
}
In Powershell:
Add-Type "" # C# Code
[CustomCertificateValidationCallback]::Install()
Consolidating and condensing some of the above learnings, I have adopted the following approach:
Syntax colored and commented like the C# of yore:
// Piggyback in System.Net namespace to avoid using statement(s)
namespace System.Net
{
// Static class to make the ps call easy
// Uses a short name that is unlikely to clash with real stuff...YMMV
public static class Util
{
// Static method for a static class
public static void Init()
{
// [optionally] clear any cruft loaded into this static scope
ServicePointManager.ServerCertificateValidationCallback = null;
// Append a dangerously permissive validation callback
// using lambda syntax for brevity.
ServicePointManager.ServerCertificateValidationCallback +=
(sender, cert, chain, errs) => true;
// Tell SPM to try protocols that have a chance
// of working against modern servers.
// Word on the street is that these will be tried from "most secure"
// to least secure. Some people add em all!
ServicePointManager.SecurityProtocol =
SecurityProtocolType.Tls |
SecurityProtocolType.Tls11 |
SecurityProtocolType.Tls12;
}
}
}
And now the real powershell highlighted version (no comments, but the same code)
Add-Type -Language CSharp #"
namespace System.Net {
public static class Util {
public static void Init() {
ServicePointManager.ServerCertificateValidationCallback = null;
ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, errs) => true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
}}}"#
[System.Net.Util]::Init()
Obviously you can remove irrelevant whitespace, but you should be able to drop that into your session, and then Invoke-WebRequest at will.
Note that the
# Do not use IMHO!
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
approach seems quite incorrect for ps 5.1 (where i have tested this). Not sure where it came from, but I wish I had avoided it and saved the heartache.
The below powershell script works for me to check post web request
add-type #"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"#
$AllProtocols = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12'
[System.Net.ServicePointManager]::SecurityProtocol = $AllProtocols
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
$uri = "XXXX"
$person = #{grant_type= 'user_password'
username = 'XXXX'
password = 'XXX'
}
$body = (ConvertTo-Json $person)
$hdrs = #{}
$hdrs.Add("XXXX","XXXX")
Invoke-RestMethod -Uri $uri -Method Post -Body $body -ContentType 'application/json' -Headers $hdrs