I am rtying to connect te WebHDFS by powershell and have been retrieving some errors. I think the 401 error is because of the Credentials.
The code I've been using is:
Invoke-RestMethod -UseDefaultCredentials -Method Put -inFile c:\Users\petter\Documents\example.txt -Uri "https://drona-haproxy.corp.com:2000/gateway/drona/webhdfs/v1/user/petter/example.txt?op=CREATE"
How could I set different parameters in the Credentials under "Invoke-RestMethod"?
THe error that I've been retrieving is:
Invoke-RestMethod : The remote server returned an error: (401) Unauthorized.
At line:1 char:1
+ Invoke-RestMethod -Method Put -inFile c:\Users\petter\Documents\load ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
You can use the Credential parameter with Invoke-RestMethod.
I would have thought you'd need to remove -UseDefaultCredentials but the Invoke-RestMethod documentation says they're part of the same parameter set - not sure how that works.
related: WebHDFS Authentication
Related
I want to use the powershell to login the website: https://uk.blsspainvisa.com/visa4spain/login, I use my Chrome can open the website, while when I use the Invoke-WebRequest to login, I get the 403 error. Any one can help me? Thanks.
enter image description here
Putting this here, since it si too long for a normal comment.
Some sites actively block some, if not all automation efforts.
So, if this site is one of them, then this is not a PowerShell issue.
You also must use TLS for all SSL-based sites.
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
Some sites, force you to go thru a landing page, and a quick test of the posted URL proves that out, as the case.
Pasting the exact URL you show in MSEdge, FF, Chrome, all present the Access Denied page. Why, because it's expecting stuff, from the previous page. Using the base URL, displays the landing page, where you must provide a country from the drop-down box first.
If you use Invoke-WebRequest on the base site, you will get denied as well,...
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest -Uri 'https://uk.blsspainvisa.com' -UseBasicParsing
Invoke-WebRequest : The remote server returned an error: (403) Forbidden.
At line:2 char:1
+ Invoke-WebRequest -Uri 'https://uk.blsspainvisa.com' -UseBasicParsing
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
...and thus all resources below it.
I'm using the google text to speech API in the powershell. I've created the project, enabled the api, created the key, when I tell it this
$cred = gcloud auth application-default print-access-token
$headers = #{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest -Method POST
-Headers $headers -ContentType: "application/json; charset=utf-8"
-InFile request.json `
-Uri "https://texttospeech.googleapis.com/v1/text:synthesize" | Select-Object -Expand Content
And this is after I set the environment, I get this error message
ERROR: (gcloud.auth.application-default.print-access-token) Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
Invoke-WebRequest : The remote server returned an error: (403) Forbidden.
At line:4 char:1
Invoke-WebRequest `
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc
eption
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
I have attempted to recreate the key thinking that was maybe the issue but I keep getting this error here no matter what I do, I am using this code to point to the key
$env:GOOGLE_APPLICATION_CREDENTIALS=C:\IVR2\IVR2-ba991f7cce6a.json
For an application, you would set the env variable GOOGLE_APPLICATION_CREDENTIALS which is the location of your service account json file.
It looks like you're using gcloud to get creds and including an authorization header. That should work too. Does your gcloud command return an error?
I have following curl request that works
curl.exe -X GET "https://host.crm.dynamics.com/api/data/v9.1/accounts?$select=name" -H "Authorization: Bearer xxxxx"
And want to implement it in PowerShell
Invoke-RestMethod "https://host.crm.dynamics.com/api/data/v9.1/accounts?`$select=name" `
-Headers #{Authorization = "Bearer xxxxx"} `
-Method Get
I've also tried to do request with Invoke-WebRequest as well as curl version in Powershell and it didn't work
Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send.
At E:\Repos\myrepo\host\connection_test.ps1:51 char:1
+ Invoke-RestMethod $hostName `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
When making connection using Invoke-RestMethod, you want to make sure that encryption your client is using is accepted by the URI.
PowerShell by default uses TLS 1.0 for web requests but the URI you are connecting to probably uses 1.1 or 1.2. You can use the following to force the use of TLS v1.1 or v1.2.
[Net.ServicePointManager]::SecurityProtocol = `
[Net.SecurityProtocolType]::Tls11, [Net.SecurityProtocolType]::Tls12;
Invoke-RestMethod "https://host.crm.dynamics.com/api/data/v9.1/accounts?`$select=name" `
-Headers #{Authorization = "Bearer xxxxx"} `
-Method Get
I'm using powershell to work with Bitbucket API. Powershell sends command to check, if pull request could be merged and then merges it.
First command works without any problems:
Invoke-WebRequest -Headers #{Authorization = "Basic $base64AuthInfo"} -Method Get -ContentType "application/json" -Uri "$BaseUrl/rest/api/1.0/projects/$ProjectKey/repos/$RepoSlug/pull-requests/$PullRequestId/merge?version=$PRVersion"
I'm getting a valid JSON response. Example:
{"canMerge":true,"conflicted":false,"outcome":"CLEAN","vetoes":[]}
The second command must use POST request in order to merge the pull request. Same command with Post instead of Get :
Invoke-WebRequest -Headers #{Authorization = "Basic $base64AuthInfo"} -Method Post -ContentType "application/json" -Uri "$BaseUrl/rest/api/1.0/projects/$ProjectKey/repos/$RepoSlug/pull-requests/$PullRequestId/merge?version=$PRVersion" -Verbose
This one returns 400 error without any other information.
VERBOSE: POST https://bitbucket.example.com/rest/api/1.0/projects/TEAM/repos/tmp-test-repo/pull-requests/8/merge?version=2 with 0-byte payload
Invoke-WebRequest : The remote server returned an error: (400) Bad Request.
At line:1 char:1
+ Invoke-WebRequest -Headers #{Authorization = "Basic $base64AuthInfo"} ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
How can make POST request work?
Reference: Bitbucket documentation
The problem is that Powershell doesn't output expanded details in Invoke-WebRequest output, so this makes investigation really hard.
Turned out that the problem was in account I was using. After I used Fiddler to get the response headers, I got a detailed response from Bitbucket API, so I was able to fix my problem.
I am trying to use the Hashicorp Atlas restful api to upload box files into but I am running into problems when uploading large files.
I am currently using the following command which works with small files like 100mb but most box files are well over 4GB where I start to see the problem:
$Filename = "c:\box\mybox.box"
$uploadpath = "https://archivist.hashicorp.com/v1/object/example"
$Timeout = 86400 #24 hours
Invoke-RestMethod -Uri $uploadPath -Method Put -InFile $Filename -TimeoutSec $Timeout -ContentType "multipart/form-data"
The error:
Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send.
At C:\Program Files\WindowsPowerShell\Modules\atlasbox\1.1.15\AtlasBox.psm1:642 char:5
+ Invoke-RestMethod -Uri $uploadPath -Method Put -InFile $Filename ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
If I use CURL command as per the documentation it works fine.
curl -X PUT --upload-file /path/to/my.box https://archivist.hashicorp.com/v1/object/example
Any idea how to make Invoke-RestMethod behave the same way curl does?
I can't take a dependency on curl because not all systems I run this on will have WSL installed and no curl.
Documentation for Atlas API is here: https://atlas.hashicorp.com/help/api/vagrant/box-providers
Looks like it might be getting weird on the SSL cert? I would try some of the recommendations here to start with, definitely an oddity specific to those cmdlets though:
Powershell v3 Invoke-WebRequest HTTPS error