How to make an authenticated web request in Powershell? - powershell

In C#, I might do something like this:
System.Net.WebClient w = new System.Net.WebClient();
w.Credentials = new System.Net.NetworkCredential(username, auth, domain);
string webpage = w.DownloadString(url);
Is there a Powershell version of this, or should I just call through to the CLR?

The PowerShell is almost exactly the same.
$webclient = new-object System.Net.WebClient
$webclient.Credentials = new-object System.Net.NetworkCredential($username, $password, $domain)
$webpage = $webclient.DownloadString($url)

For those that need Powershell to return additional information like the Http StatusCode, here's an example. Included are the two most likely ways to pass in credentials.
Its a slightly modified version of this SO answer:
How to obtain numeric HTTP status codes in PowerShell
$req = [system.Net.WebRequest]::Create($url)
# method 1 $req.UseDefaultCredentials = $true
# method 2 $req.Credentials = New-Object System.Net.NetworkCredential($username, $pwd, $domain);
try
{
$res = $req.GetResponse()
}
catch [System.Net.WebException]
{
$res = $_.Exception.Response
}
$int = [int]$res.StatusCode
$status = $res.StatusCode
return "$int $status"

In some case NTLM authentication still won't work if given the correct credential.
There's a mechanism which will void NTLM auth within WebClient, see here for more information: System.Net.WebClient doesn't work with Windows Authentication
If you're trying above answer and it's still not working, follow the above link to add registry to make the domain whitelisted.
Post this here to save other's time ;)

Related

PowerShell System.Net.WebClient never closes ftp connection

I'm trying to use PowerShell to upload a (long) list of queued files, using System.Net.WebClient and the UploadFile function. This works fine, but after a file us uploaded, the ftp-connection never closes, either when the WebClient object instance goes out of scope or even after the script has finished. The function looks as follows:
function Upload-File() {
Param (
[string] $user,
[string] $password,
[string] $srceFileName,
[string] $destFileName
)
# Set up FTP-client
$client = New-Object System.Net.WebClient
$client.Credentials = New-Object System.Net.NetworkCredential($user, $password)
$client.UploadFile($destFileName, ".\$srceFileName")
$client.Dispose()
}
All the information I can find states that the connection should close automatically when $client goes out of scope but this is clearly not happening.
Any idea on how to force the connection to close?
(This is part of a legacy system and for now I am stuck with ftp, so switching to another protocol is not an option.)
For anyone else running into this problem, the solution is to use FtpWebRequest instead of WebClient and to set KeepAlive = $false. The function below will upload and then terminate the connection immediately afterwards.
function Upload-File() {
Param (
[string] $user,
[string] $password,
[string] $srceFileName,
[string] $destFileName
)
$request = [Net.WebRequest]::Create($destFileName)
$request.KeepAlive = $false
$request.Credentials =
New-Object System.Net.NetworkCredential($user, $password)
$request.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$fileStream = [System.IO.File]::OpenRead(".\$srceFileName")
$ftpStream = $request.GetRequestStream()
$fileStream.CopyTo($ftpStream)
$ftpStream.Dispose()
$fileStream.Dispose()
}
This post pointed me in the right direction.

Powershell - How to receive large response-headers from error-response of a web-request?

I am looking for a solution to parse an error-response of a given web-service.
Below sample works great in general, but if the response is larger than 64kb then the content is not availabe in the exception at all.
I have seen some solutions recommending to use webHttpClient and increase the MaxResponseContentBufferSize here, but how can I do this for a given WebClient-object?
Is there any option to change that BufferSize globally for all net-webcalls like below TLS12-settings?
Here is my sample-code:
# using net-webclient to use individual user-side proxy-settings:
$web = new-object Net.WebClient
[Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$url = "address to web-service"
try {
$response = $web.DownloadString($url)
} catch [System.Net.WebException] {
# this part needs to work even if the error-response in larger than 64kb
# unfortunately the response-object is empty in such case
$message = $_.Exception.Response
$stream = $message.GetResponseStream()
$reader = new-object System.IO.StreamReader ($stream)
$body = $reader.ReadToEnd()
write-host "#error:$body"
}
I solved it at the end by switching to system.net.httpclient.
That way I still repect any custom proxy-settings and also avoid the above mentioned 64kb-limit in any error-response. Here a sample how to use it:
$url = "address to web-service"
$cred = Get-Credential
# define settings for the http-client:
Add-Type -AssemblyName System.Net.Http
$ignoreCerts = [System.Net.Http.HttpClientHandler]::DangerousAcceptAnyServerCertificateValidator
$handler = [System.Net.Http.HttpClientHandler]::new()
$handler.ServerCertificateCustomValidationCallback = $ignoreCerts
$handler.Credentials = $cred
$handler.PreAuthenticate = $true
$client = [System.Net.Http.HttpClient]::new($handler)
$client.Timeout = [System.TimeSpan]::FromSeconds(10)
$result = $client.GetAsync($url).result
$response = $result.Content.ReadAsStringAsync().Result
write-host $response

Powershell: Net.Webclient - not getting reply from intranet depending on machine

Cheers everyone,
I am getting the weirdest problem for which I need your helping ideas how to approach the issue.
So, I have a download script that pulls content off a company intranet using Webclient objects. It requires credentials and it is working on about 80% of the computers. The script pulls a listing using .DownloadString and then parses and gets some files using .DownloadFile.
On the machines that won't work the initial .DownloadString hangs until it appears to run into a timeout and returns $null.
User credentials are irrelevant on these types of machines meaning a user that works on another machine fails on this one.
Addresses, if entered into browser returns content.
Spoken in code I try it this way:
$wc = new-object System.Net.WebClient
$wc.Credentials = new-object System.Net.NetworkCredential($user, $pass, $domain)
$old_eap = $ErrorActionPreference
$ErrorActionPreference = "Stop"
try
{
$tmp = $wc.DownloadString($url)
if ([String]::IsNullOrEmpty($tmp))
{
throw "Intranet server did not return directory listing"
}
Return $tmp #the code is actually part of a function...
}
catch
{
write-error $_.Exception.Message
Return $null
}
finally
{
$ErrorActionPreference = $old_eap
}
I have no idea other than looking for changed settings between different machines. But which settings could be relevant for Webclient behaving like this? Any Ideas? I am seriously stuck...
I forgot... To make things a little easier I am stuck with Version 2.0 and we cant update yet. Bummer...
Thanks in advance
Alex
Maybe try to use xmlhttp as a client. Below is the usage example.
$url = "https://example.com/"
$http = New-Object -ComObject Msxml2.XMLHTTP
$user = "Domain\username"
$pwd = "password"
$utf = [System.Text.Encoding]::UTF8
$http.open("GET", $url, $false, $user, $pwd)
$http.send()
$result = $utf.GetString($http.responseBody)

Trying to do a simple post request in powershell v2.0, no luck

I'm simply trying to do a HTTP POST request with some keys and values. I can't get this to work for the life of me and yes I know this should be simple.
Here's what I've tried:
$Body = [byte[]][char[]]'username=asdf';
$Request = [System.Net.HttpWebRequest]::CreateHttp('http://mysite/test.php');
$Request.Method = 'POST';
$Stream = $Request.GetRequestStream();
$Stream.Write($Body, 0, $Body.Length);
$Request.GetResponse();
This doesn't work in Powershell v2.0 because I get the error
Method
invocation failed because [System.Net.HttpWebRequest] doesn't contain
a method named 'CreateHttp'.
Next, I've taken someone else's example of:
$URI1 = "http://mysite/test.php"
$request = [System.Net.WebRequest]::Create($URI1)
$request.ContentType = "application/xml"
$request.Method = "POST"
$body = "username=test"
# $request | Get-Member for a list of methods and properties
try
{
$requestStream = $request.GetRequestStream()
$streamWriter = New-Object System.IO.StreamWriter($requestStream)
$streamWriter.Write($body)
}
finally
{
if ($null -ne $streamWriter) { $streamWriter.Dispose() }
if ($null -ne $requestStream) { $requestStream.Dispose() }
}
$res = $request.GetResponse()
but for some reason "username" doesn't get noticed when test.php echos $_POST['username']
Can someone please help tell me what I'm missing here? I've been googling for hours and everything I try isn't working for some reason. Works fine on Powershell versions greater than 2.0, but not 2.0 (default in Windows 7).
Invoke-WebRequest and Invoke-RestMethod do not work on Powershell v2.0, so I'm forced to find all these annoying alternatives.
* EDIT *
I got it working after finding another HTTP POST request example:
$url = "http://mysite/test.php"
$postData = "username=test"
$buffer = [text.encoding]::ascii.getbytes($postData)
[net.httpWebRequest] $req = [net.webRequest]::create($url)
$req.method = "POST"
$req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
$req.Headers.Add("Accept-Language: en-US")
$req.Headers.Add("Accept-Encoding: gzip,deflate")
$req.Headers.Add("Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7")
$req.AllowAutoRedirect = $false
$req.ContentType = "application/x-www-form-urlencoded"
$req.ContentLength = $buffer.length
$req.TimeOut = 50000
$req.KeepAlive = $true
$req.Headers.Add("Keep-Alive: 300");
$reqst = $req.getRequestStream()
$reqst.write($buffer, 0, $buffer.length)
$reqst.flush()
$reqst.close()
[net.httpWebResponse] $res = $req.getResponse()
$resst = $res.getResponseStream()
$sr = new-object IO.StreamReader($resst)
$result = $sr.ReadToEnd()
$res.close()
This was discovered on another site before additional comments came here; however, I've tried the solutions from people's suggestions below and also was able to get this working.
In the first example code, the method should be named Create, Not CreateHttp
In the second block of code, you set the content-type to 'application/xml', but the body is plain-text.
This method, and your initial example itself, would work in a newer version of PowerShell. Try installing dotnet 4.5 on your system, then WMF 4.0, and this should work with no issue.
The dotnet class of [System.Net.HttpWebRequest] didn't have the static method CreateHttp() until dotnet 4.5, as seen here on MSDN Docs.
Why not just update this one machine to a newer version of PowerShell? It will be a LOT less pain.

Powershell http post with .cer for auth

Pretty new to PowerShell and hoping someone can point me in the right direction.
I need to perform a POST request and have to pass it a locally stored cert (x509) during the POST request, for authentication.
What is the best way or way to accomplish this? I've found plenty of example to be able to perform this task in .net/C# but I am not finding anything that will accomplish this task in PowerShell.
Here is my POST request code, again I would like to point to a cert stored locally "C:\code\cert.crt" and pass it during the web transaction.
$url = "https://myUrl/uploadTester"
$data = '{"data": "988309487577839444"}'
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$b = [System.Text.Encoding]::ASCII.GetBytes($data)
$web = [System.Net.WebRequest]::Create($url)
$web.Method = "POST"
$web.ContentLength = $b.Length
$web.ContentType = "application/x-www-form-urlencoded"
$stream = $web.GetRequestStream()
$stream.Write($b,0,$b.Length)
$stream.close()
$reader = New-Object System.IO.Streamreader -ArgumentList $web.GetResponse().GetResponseStream()
$reader.ReadToEnd()
$reader.Close()
Thanks for all the help in advanced.
It's pretty easy to convert C# to PowerShell. Give this a try:
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$cert = [System.Security.Cryptography.X509Certificates.X509Certificate2]::CreateFromCertFile("C:\Users\Andy\Desktop\Test.cer")
$web = [System.Net.WebRequest]::Create($url)
$web.ClientCertificates.Add($Cert)
I adapted this from: http://support.microsoft.com/kb/895971
Looks like the key is the ClientCertificates property.
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.clientcertificates.aspx