Powershell HTTP Post Form-Data upload file [duplicate] - powershell

We have a php site that uploads pictures to server, simple jpg file with correct naming. Problem is that we need to upload sometimes couple of hundred of them at a time, but php accepts only 1 at a time, renames them and uploads. I have done file operations in PS quite well, but fail to upload.
PHP part related to upload (as far as I can tell) looks like this: <form name='' id='' enctype='multipart/form-data' method='POST' action='/picture_upload.php' target='_self' onsubmit="default_on_submit(event)">
I checked Google, related topics here as well, and got to this:
$uri = "http://example.com/"
$pwd = ConvertTo-SecureString 'MyPassword' -AsPlainText -Force
$cred = New-Object Management.Automation.PSCredential ('myuser', $pwd)
$contentType = "multipart/form-data"
$body = #{
"FileName" = Get-Content($uploadPath) -Raw
}
Invoke-WebRequest -Uri $uri -Method Post -ContentType $contentType -Body $body
I have checked $uploadPath and it is correct C:\Folder\file.jpg. I use credentials that I use to log in to site where I can upload these pictures via GUI.
I have tried switching between POST and PUT, with no changes.
Replacing http://example.com with http://example.com/file.jpg also provided no difference. Unsure, which is correct way to use POST.
We have McAffe web gateway in company, but I'm running script with user that bypasses it, so it's not causing this.
Current error message that I'm getting is:
"Invoke-WebRequest : The underlying connection was closed: An unexpected error occurred on a receive."
Any help would be greatly appreciated! And sorry if this has been already solved and I've simply missed an entry!
P.S. I have also tried this - Powershell script to Upload log file from local system to http URL, and it returns Exception calling "UploadFile" with "3" argument(s): "An exception occurred during a WebClient request."

A year late, so you probably don't need the answer anymore, but what worked for me was:
Invoke-RestMethod -Uri $uri -Method Post -InFile $uploadPath -UseDefaultCredentials
Mine needed to use windows auth as the currently logged in user... From this other answer, looks like you can use -cred in some cases and have to build out an Authorization header in others.

Even later post but these Invoke-RestMethod solutions (and a few others) didn't work for me. My service endpoint was failing with missing boundaries and other "content" issues, depending on my trial and error.
I tried using WebClient's Uploadfile() functionality and it worked for me.
Thankfully, it's concise.
$wc = New-Object System.Net.WebClient
$resp = $wc.UploadFile($uri,$uploadPath)

Related

Getting a file from BitBucket Rest API v2.0

I have a script which grabs a file from GIT using the bitbucket REST API (1.0) however it has recently stopped working. I'm theorizing this may be due to the v1 REST API being depreciated but I'm not sure.
Anyway I am trying to retrieve the file using the new 2.0 REST API but I can't seem to get the syntax right as the request continually fails.
I'm starting out with curl since its easiest to test. This is what I'm trying:
curl -u myusername#mydomain.com "https://api.bitbucket.org/2.0/repositories/MyCompany/myrepo/downloads/Scripts/Environment Setup/test.txt"
Enter host password for user 'myusername#mydomain.com': redacted
{"type": "error", "error": {"message": "Resource not found", "detail": "There is no API hosted at this URL.\n\nFor information about our API's, please refer to the documentation at: https://developer.atlassian.com/bitbucket/api/2/reference/"}}
Here is the reference documentation I am using: https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/downloads/%7Bfilename%7D
Maybe I am using the wrong function? I'm not sure.
For posterities sake, you don't want to use the following to download an individual file from bitbucket:
https://api.bitbucket.org/2.0/repositories/MyCompany/myrepo/downloads/path/to/your/file.txt
("Downloads" is to download entire repo files like a .zip file)
Instead you want to do:
curl --user myuser#mydomain.com:password "https://api.bitbucket.org/2.0/repositories/MyCompany/myrepo/src/master/path/to/file.txt"
If you're trying to use Invoke-RestRequest (in powershell) note there are some extra steps. With the old 1.0 API you could do:
$cred = Get-Credential
$uri = "https://api.bitbucket.org/1.0/repositories/MyCompany/$($filepath)"
# Get the files from bitbucket (GIT)
Invoke-RestMethod -Credential $cred -Uri $uri -Proxy $proxyUri -OutFile $destination
With the new 2.0 API that no longer works. Powershell's Invoke-RestMethod waits for a 401 response before sending the credentials, and the new 2.0 bitbucket api never provides one, so credentials never get sent causing a 403 forbidden.
To work around that you have to use the following ugly hack to force Invoke-RestMethod to send the credentials immediately in an Authorization header:
$cred = Get-Credential
$uri = "https://api.bitbucket.org/2.0/repositories/MyCompany/$($filepath)"
$username = ($cred.GetNetworkCredential()).username
$password = ($cred.GetNetworkCredential()).password
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
# Get the files from bitbucket (GIT)
Invoke-RestMethod -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Uri $uri -Proxy $proxyUri -OutFile $destination
Hopefully that helps someone else out in the future!
Thanks #Jim Redmond for the help.
You can also use the PowerShell module BitbucketServerAutomation. There's not a ton of cmdlets, they do have Get-BBServerFile and Get-BBServerFileContent. I have found it is well written, very usable and being updated regularly. The Invoke-BBServerRestMethod cmdlet is available if you need a command it doesn't have.

PowerShell Invoke-Webrequest not compatible with site?

I've seen many examples of Invoke-Webrequest and I've already had some success with it myself, however, one site where I was trying to automate my login just hasn't worked no matter what I try. Here is the code:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 # This is required so that HTTPS requests won't fail with Invoke-WebRequest
$r = Invoke-WebRequest -Uri https://order.swisschalet.com -SessionVariable sc
$r.forms[0].fields['form-login-header-email'] = "MyEmail"
$r.forms[0].fields['form-login-header-password'] = 'MyPassword'
$a = Invoke-WebRequest -Uri ("https://order.swisschalet.com" + $r.forms[0].Action) -WebSession $sc -Method POST -Body $r.forms[0]
I have tried using Fiddler 4 to analyze what is going on but it has only confused me even more. When I manually go to the website Fiddler shows 'email' and 'password' fields that were posted rather than what originally came back in the forms which is 'form-login-header-email' and 'form-login-header-password'. However, even if I try to create these new fields and POST them it still doesn't work. Fiddler shows that going to the website manually also creates some kind of synchronization token called 'org.codehaus.groovy.grails.SYNCHRONIZER_TOKEN'.
I am beginning to wonder if Invoke-WebRequest is simply incompatible with this site as I can never get the expected response where I can find my name in the $a.parsedHTML.DocumentElement.InnerText. Instead, when I view this I simply get the full page back telling me that my session has already expired.
I started to try this with the IE Com Object as well but this also did not seem to work. Am I missing something or is it just the way this site has been made? I've been struggling with this (really just to learn) for a couple of days now.
Thanks for any help!
If you send what it sends, to the place it sends it, you get logged in:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 # This is required so that HTTPS requests won't fail with Invoke-WebRequest
$r = Invoke-WebRequest -Uri https://order.swisschalet.com -SessionVariable sc
$form = #{
delegate='login'
id='null'
mode='save'
target='#form-login'
email='email#example.com'
password='password'
}
$a = Invoke-WebRequest -Uri "https://order.swisschalet.com/order/auth/index" -WebSession $sc -Method POST -Body $form
and it says
<span class="welcome-back">Welcome, <span class="username">TestingSome StuffForStackOverflo</span>
in the $a.RawContent result. So I guess it's not incompatible.

Uploading file to HTTP via Powershell

We have a php site that uploads pictures to server, simple jpg file with correct naming. Problem is that we need to upload sometimes couple of hundred of them at a time, but php accepts only 1 at a time, renames them and uploads. I have done file operations in PS quite well, but fail to upload.
PHP part related to upload (as far as I can tell) looks like this: <form name='' id='' enctype='multipart/form-data' method='POST' action='/picture_upload.php' target='_self' onsubmit="default_on_submit(event)">
I checked Google, related topics here as well, and got to this:
$uri = "http://example.com/"
$pwd = ConvertTo-SecureString 'MyPassword' -AsPlainText -Force
$cred = New-Object Management.Automation.PSCredential ('myuser', $pwd)
$contentType = "multipart/form-data"
$body = #{
"FileName" = Get-Content($uploadPath) -Raw
}
Invoke-WebRequest -Uri $uri -Method Post -ContentType $contentType -Body $body
I have checked $uploadPath and it is correct C:\Folder\file.jpg. I use credentials that I use to log in to site where I can upload these pictures via GUI.
I have tried switching between POST and PUT, with no changes.
Replacing http://example.com with http://example.com/file.jpg also provided no difference. Unsure, which is correct way to use POST.
We have McAffe web gateway in company, but I'm running script with user that bypasses it, so it's not causing this.
Current error message that I'm getting is:
"Invoke-WebRequest : The underlying connection was closed: An unexpected error occurred on a receive."
Any help would be greatly appreciated! And sorry if this has been already solved and I've simply missed an entry!
P.S. I have also tried this - Powershell script to Upload log file from local system to http URL, and it returns Exception calling "UploadFile" with "3" argument(s): "An exception occurred during a WebClient request."
A year late, so you probably don't need the answer anymore, but what worked for me was:
Invoke-RestMethod -Uri $uri -Method Post -InFile $uploadPath -UseDefaultCredentials
Mine needed to use windows auth as the currently logged in user... From this other answer, looks like you can use -cred in some cases and have to build out an Authorization header in others.
Even later post but these Invoke-RestMethod solutions (and a few others) didn't work for me. My service endpoint was failing with missing boundaries and other "content" issues, depending on my trial and error.
I tried using WebClient's Uploadfile() functionality and it worked for me.
Thankfully, it's concise.
$wc = New-Object System.Net.WebClient
$resp = $wc.UploadFile($uri,$uploadPath)

Powershell and NITRO API for Citrix NetScaler error on GET method

I am using a PowerShell module provided by Citrix to invoke the Nitro REST API. Calling the function I can successfully add and remove load balanced services from the load. However when I try to do a GET method to get the status of a service I get the error:
Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send.
I have tried running Invoke-RestMethod without using the module but get the same error
Invoke-RestMethod -WebSession $myNSSession.WebSession -Method GET -Uri https://<NetScaler IP/nitro/v1/config/service/<Service Name>
When googling this error everything seems to point to certificate issues. I had this initially even on POST method until i added the below to my script
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
So since this works for doing POST i cant see why it wouldn't for a GET!!
another weird thing is, if I put the URL directly into the browser then enter my credentials i get a response in raw text! so it looks like this is an issue with the way i am calling it in PowerShell rather than the NetScaler or the NITRO API!
Someone please help as this is driving me crazy!!
Admitedly i am new to Invoke-RestMethod commands, but try this:
$creds = Get-Credential
$service = Invoke-RestMethod -Uri https://<NetScaler IP/nitro/v1/config/service/<Service Name> -Credential $creds
What you will get is something similar to this:
*errorcode* *message* *serverity* *service*
* 0 Done NONE {#{name=<service name; n..
then type $service.service and you will see more information. whatever attributes are availible will be listed. then just follow the pattern:
$service.service.
I had the same problem with Nitro API (specifically v10.5), and found that setting certificate policies, TLS versions and trust settings had no effect. POST works, GET fails.
The solution for me was to not use the cmdlets and instead drop back to a native .Net method. Below I am still using HTTPS with an internal certificate, hence still setting the callback.
$NSProtocol = "https://"
$NSHostname = "netscaler"
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$WebRequest = [System.Net.WebRequest]::Create("$NsProtocol$NsHostname/nitro/v1/config/hanode")
$WebRequest.Method = "GET"
$WebRequest.ContentType = "application/json; charset=utf-8";
$WebRequest.Headers.Add("AUTHORIZATION","Basic $([System.Convert]::ToBase64String([System.Text.Encoding]::GetEncoding("ISO-8859-1").GetBytes($nsuser+":"+$nspass)))")
$Response = $WebRequest.GetResponse()
$ReadStream = New-Object System.IO.StreamReader $Response.GetResponseStream()
$HaState = ConvertFrom-Json $ReadStream.ReadToEnd()
Hope that helps.

Powershell Invoke-Webrequest post method - incorrect payload length

I am trying to submit a form using invoke-webrequst cmdlet, and this is the code
$postParams = #{regno='1234567';dob='01/01/1997';B1='Get Marks'}
$response = Invoke-Webrequest -Uri ("http://studentresulsts/res.asp") -Body $postParams -Method Post -Debug -OutFile out.html
VERBOSE: POST http://studentresulsts/res.asp with -1-byte payload
VERBOSE: received 13-byte response of content type text/html
The $response comes back as 'Access Denied' (13-byte response)
The payload length is shown as 1-byte while $postParams is clearly more than that. Wondering if that's the reason I am getting 'Access Denied'.
Checked the form manually in browser and it works fine with correct field values.
I am using powershell 4.0
Answer : Okay, I was missing referrer URL in the header which the server was looking for, then I included referrer and it works fine :) . My new script looks like this. Thanks for your help...
$postParams = #{regno='1234567';dob='01/01/1997';B1='Get Marks'}
$headervals = #{'Referer'='http://studentresulsts/gdslplus/gdslform.htm';'Content-Type'='application/x-www-form-urlencoded'}
$response = Invoke-Webrequest -Uri ("http://studentresulsts/res.asp") -Body $postParams -Method Post -Debug -OutFile out.html
The fact that you don't have to manually authenticate does not mean the access isn't verified(e.g. trusted sites in IE).
Perhaps you are missing a crucial header?