Unable to ingest JSON stream data to Azure Event Hub - powershell

I would like to post the JSON results which I am getting from an API endpoint to the Azure Event Hub $default consumer group but I am getting the below error:
Invoke-RestMethod : The remote server returned an error: (401) Unauthorized.
At H:\Users\User1 - scripts\get_mel_streetparking_data.ps1:34 char:1
+ Invoke-RestMethod -Uri $URI -Method $method -Headers $headers -Body $res ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke- RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I wrote the below piece of powershell script where the following piece of get code runs fine:
$url = "https://data.melbourne.vic.gov.au/resource/vh2v-4nfs"
$apptoken = "abcdasdadaaf"
# Set header to accept JSON
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept","application/json")
$headers.Add("X-App-Token",$apptoken)
$results = Invoke-RestMethod -Uri $url -Method get -Headers $headers
but below piece of code fails with the above given error:
$method = "POST"
$URI = "https://[servicebusNamespace].servicebus.windows.net/[eventHubPath]/messages"
$signature = "SharedAccessSignature sr=[servicebusNamespace].servicebus.windows.net%2feventhub- streetparking&sig=%3dgZfDHEGN8lVEGgqu4N64TW70BLuSKARSKgMPeRByc%5d&se=604985&skn=RootManageSharedAccessKe"
# API headers
$headers = #{
"Authorization"=$signature;
"Content-Type"="application/json";
}
# create Request Body
#$body = "$results"
# execute the Azure REST API
Invoke-RestMethod -Uri $URI -Method $method -Headers $headers -Body $results
I generated SAS token with the help of the below code:
[Reflection.Assembly]::LoadWithPartialName("System.Web")| out-null
$URI="[servicebusNamespace].servicebus.windows.net/[eventHubPath]"
$Access_Policy_Name="RootManageSharedAccessKey"
$Access_Policy_Key="Root key value"
#Token expires now+3000
$Expires=([DateTimeOffset]::Now.ToUnixTimeSeconds())+3000
$SignatureString=[System.Web.HttpUtility]::UrlEncode($URI)+ "`n" + [string]$Expires
$HMAC = New-Object System.Security.Cryptography.HMACSHA256
$HMAC.key = [Text.Encoding]::ASCII.GetBytes($Access_Policy_Key)
$Signature = $HMAC.ComputeHash([Text.Encoding]::ASCII.GetBytes($SignatureString))
$Signature = [Convert]::ToBase64String($Signature)
$SASToken = "SharedAccessSignature sr=" + [System.Web.HttpUtility]::UrlEncode($URI) + "&sig=" + [System.Web.HttpUtility]::UrlEncode($Signature) + "&se=" + $Expires + "&skn=" + $Access_Policy_Name
$SASToken
Please help me why am I seeing unauthorized error

It was my bad that I did not replace the below values before the SAS token generation.
$Access_Policy_Name="RootManageSharedAccessKey"
$Access_Policy_Key="Root key value"

Related

RestAPI authentication with PowerShell

I am trying to access the API of OpenProvider using PowerShell and I can't seem to get past Authentication.
The documentation for the API is here : https://support.openprovider.eu/hc/en-us/articles/360025683173-Getting-started-with-Openprovider-API
And my code looks like this:
$EndPoint = "https://api.openprovider.eu/v1beta/auth/login"
function Get-ConfHeaders
{
##Configure headers
$Headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$Headers.Add("ip","0.0.0.0")
$Headers.Add("username","myusername")
$Headers.Add("hash","APIpasswordhashgoeshere")
return $Headers
}
$header = Get-ConfHeaders
Invoke-RestMethod -Method Post -Uri $EndPoint -Headers $header
The response get is :
Invoke-RestMethod : The remote server returned an error: (500) Internal Server Error.
At line:36 char:1
+ Invoke-RestMethod -Method Post -Uri $EndPoint -Headers $header
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I am by no means an expert when it comes to API and any help would be appreciated.
Ok, I think the API documentation here leaves a lot to be desired.
You need to include the authentication in the body and it needs to be converted to JSON format. So the working code looks like this:-
$EndPoint = "https://api.openprovider.eu/v1beta/auth/login"
function Get-ConfHeaders
{
##Configure headers
$Headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$Headers.Add("ip","0.0.0.0")
$Headers.Add("username","username")
$Headers.Add("password","passwordhere")
return $Headers
}
$header = Get-ConfHeaders | ConvertTo-Json
Invoke-RestMethod -Method Post -Uri $EndPoint -body $header -ContentType 'application/json'
Thanks for the help everyone.

POST method in powershell gives 405 error

I have a POST method that works fine in Postman but I can't get it working in powershell.
The client id and client secret are setup in AWS cognito.
I am using basic authentication to retrieve the access token.
Below is my code,
$encodedlogin="MNBvbmZfYWsxUTRocG1qOG5zOGZqZ8xvN646a1plMDNsMTZjY6xucTA5MzVkOGg2aGdlaG1mdXAxaHZkMmlxaXY0PKOdM3BcxNBtPLs5"
$authorisation = "Basic " + $encodedlogin
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization",$authorisation)
$headers.Add("Accept","application/json")
$headers.Add("Content-Type","application/json")
$headers.Add("ResponseType","id_token")
$body = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$body.Add("grant_type","client_credentials")
$body.Add("scope","cybergate-gateway-resources/cybertron.fullaccess")
$uri = "https://cybergate.auth.eu-west-2.amazoncognito.com/oauth2/token"
$response = Invoke-WebRequest -Uri $uri -Headers $headers -body $body -Method Post -ContentType "application/json"
The error as follows,
Invoke-WebRequest : The remote server returned an error: (405) Method Not Allowed.
At C:\CyberGate\src\CyberGate.API\Scripts\Deploy\CyberGate.ps1:33 char:17
+ ... $response = Invoke-WebRequest -Uri $uri -Headers $headers -body $body ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
Any ideas?
Thanks

Team City build step with Powershell Invoke-RestMethod fails if URI contains URL encoded value

I have a build step in TeamCity which sends an HTTP request, like this:
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Token", "%TOKEN%")
$uri = 'https://host/apps/branches/%BRANCH_NAME%/builds'
$response = Invoke-RestMethod $uri -Method 'GET' -Headers $headers
Write-Host "##teamcity[setParameter name='url' value='$uri']"
It works perfectly when %BRANCH_NAME% actually contains simple branch names, like ' master', 'beta', 'dev'.
But it works not so good when %BRANCH_NAME% is release%2F2019-DEC (URL encoded value of release/2019-DEC)
I tried the same script on local Powershell, and it worked flawlessly, but when I run it as TeamCity build step I receive the following error:
Invoke-RestMethod : The remote server returned an error: (404) Not Found.
At C:\TeamCityBuildAgent4\temp\buildTmp\powershell7367326510068615884.ps1:5 char:13
+ $response = Invoke-RestMethod $uri -Method 'GET' -Headers $headers
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc
eption
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
##teamcity[setParameter name='url' value='https://host/app/branches/release%2F2019-DEC/']
I tried to escape the percent sign using the extra % sign as in Using the percent sign in TeamCity build scripts, but unfortunately, it returns the same error.
Any ideas?
Solved:
Seemingly issue was caused by the old version of PowerShell on Agent, which caused the behavior described in Team City build step with Powershell Invoke-RestMethod fails if URI contains URL encoded value
That means that no extra percent signs are actually needed, we can URL encode value inside the script.
So the final version will look like:
function fixuri($uri){
$UnEscapeDotsAndSlashes = 0x2000000;
$SimpleUserSyntax = 0x20000;
$type = $uri.GetType();
$fieldInfo = $type.GetField("m_Syntax", ([System.Reflection.BindingFlags]::Instance -bor [System.Reflection.BindingFlags]::NonPublic));
$uriParser = $fieldInfo.GetValue($uri);
$typeUriParser = $uriParser.GetType().BaseType;
$fieldInfo = $typeUriParser.GetField("m_Flags", ([System.Reflection.BindingFlags]::Instance -bor [System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::FlattenHierarchy));
$uriSyntaxFlags = $fieldInfo.GetValue($uriParser);
$uriSyntaxFlags = $uriSyntaxFlags -band (-bnot $UnEscapeDotsAndSlashes);
$uriSyntaxFlags = $uriSyntaxFlags -band (-bnot $SimpleUserSyntax);
$fieldInfo.SetValue($uriParser, $uriSyntaxFlags);
}
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("X-API-Token", "%TOKEN%")
$branch = [System.Web.HttpUtility]::UrlEncode('release/2019-DEC1')
$uri = New-Object System.Uri -ArgumentList ("https://host/apps/branches/$branch")
fixuri $uri
$response = Invoke-RestMethod $uri -Method 'GET' -Headers $headers
Special thanks to #Peska

How to attach CSV file to Service Now incident via REST API using PowerShell?

I need to attach the file either xlsx or CSV to a particular incident via SNOW REST API using PowerShell script. I have tried with the below code:
if (!$script:ServiceNowCreds) {
$script:ServiceNowCreds = Get-Credential
}
$snow_url = 'https://dev652xx.service-now.com/api/now/table/incident'
$Body = #{
'number' = 'INC00xx059'
}
$result = Invoke-RestMethod -Uri $snow_url -Credential $script:ServiceNowCreds -Body $Body -ContentType "application/json"
$result.result | select sys_id, number | ForEach-Object {
$Upload_snow_url ='https://dev652xx.servicenow.com/api/now/attachment/upload'
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add('Content-Type','text/csv')
$headers.Add('Accept','*/*')
$sys_id = $_.sys_id
$incident_number = $_.number
$UploadBody = #{
'table_name'='incident';
'table_sys_id'=$sys_id;
'file_name' = 'C:\Users\suganthanraj.p\Documents\Servers.csv'
}
$uploadParam = $UploadBody | ConvertTo-JSon
Write-Host $sys_id
Write-Host $incident_number
$UploadResult = Invoke-RestMethod -Uri $Upload_snow_url -Credential $script:ServiceNowCreds -Body $uploadParam -Method Post -Headers $headers
$UploadResult
}
When I execute the above script I am getting the below error:
Invoke-RestMethod : The remote server returned an error: (415) Unsupported
Media Type.
At C:\Users\suganthanraj.p\Desktop\SNOW-UploadAttachment.ps1:39 char:21
+ ... oadResult = Invoke-RestMethod -Uri $Upload_snow_url -Credential $scr ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
Try changing you content type to "multipart/form-data"
$headers.Add('Content-Type','multipart/form-data')
$UploadBody = #{
'table_name'='incident';
'record_sys_id'=$sys_id;
'uploadFile' = 'C:\Users\suganthanraj.p\Documents\Servers.csv'
}
The error says "The remote server returned an error: (415) Unsupported
Media Type."
Doco on the api can be found here:
https://docs.servicenow.com/bundle/geneva-servicenow-platform/page/integrate/inbound_rest/reference/r_AttachmentAPI-POSTmultipart.html
Your best option would be leverage the OOB Attachment API in ServiceNow. You will need to make a post call from powershell. Powershell has two options for this Invoke-RestMethod and Invoke-WebRequest. I have had better luck with the latter when trying to POST. You might also first build your rest call in Postman make sure you can get the attachment into ServiceNow, then worry about writing your PS.
$Body = #{
User = 'jdoe'
password = 'P#S$w0rd!'
}
$LoginResponse = Invoke-WebRequest 'http://www.contoso.com/login/' - SessionVariable 'Session' -Body $Body -Method 'POST'
$Session
$ProfileResponse = Invoke-WebRequest 'http://www.contoso.com/profile/' -`WebSession $Session $ProfileResponse`
Finally i found answer from the below link
https://community.servicenow.com/community?id=community_question&sys_id=d3707023dbaceb8023f4a345ca961949 and below is the code:
# Eg. User name="admin", Password="admin" for this code sample.
$user = "admin"
$pass = "XXX"
# Build auth header
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user, $pass)))
# Set proper headers
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add('Authorization',('Basic {0}' -f $base64AuthInfo))
$headers.Add('Accept','application/json')
$headers.Add('Content-Type','application/json')
# Specify endpoint uri
$uri = "https://dev652XX.service-now.com/api/now/attachment/file?table_name=incident&table_sys_id=850XXXXX2200e0ef563dbb9a71c1&file_name=TreeSizeReport.csv"
# Specifiy file to attach
$fileToAttach = "C:\Users\suganthanraj.p\Desktop\TreeSizeReport.csv"
# Specify HTTP method (POST, PATCH, PUT)
$method = "POST"
# Send HTTP request
$response = Invoke-WebRequest -Headers $headers -Method $method -Uri $uri -InFile $fileToAttach
# Print response
$response.RawContent

Invoke-RestMethod to Post API Error

Tearing my hair out with this, i'm trying to use the kanboard API (kanboard.org) from powershell, essentially using curl which from investigation is exactly what Invoke-RestMethod is.
I'm getting the following error;
Invoke-RestMethod : Cannot bind parameter 'Headers'. Cannot convert the ""X-API-Auth",
"agBzAG8AbgByAHAAYwA6ADQAMgBjAGQANwAwADEAMgA3ADgAMgAyAGMAYwBiAGUANwA1ADcAMAA3AGYANQBmAGUAZQA2ADkANgBmADYAYQA1AGIAOQAwADIANABiADEAZQBhADAANwBjAGIAOQA1ADQAYQA5AGQAYQA5AGMAYQAwAGYAYgA0AA=="" value of type
"System.String" to type "System.Collections.IDictionary".
At line:1 char:77
+ ... RestMethod -Uri $kanboardserver -Method Post -Headers $header -Body $ ...
+ ~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Invoke-RestMethod], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
My API call code using powershell is this;
$kanboardserver = "http://mykanboardserver/jsonrpc.php"
$kanboardtoken = "jsonrpc:42cd70127822ccbe75707f5fee696f6a5b9024b1ea07cb954a9da9ca0fb4"
$converted = $kanboardtoken | ConvertTo-Base64
$headersKan = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" -ErrorAction Stop
$headersKan.Add("X-API-Auth", $converted)
$payload = #{ jsonrpc = "2.0";
method = "getAllProjects";
id = 1;
} | ConvertTo-Json
$responseKan = Invoke-RestMethod -Uri $kanboardserver -Method Post -Headers $header -Body $payload -ErrorAction Stop
Write-Host $responseKan
Just for reference this is the documentation example of how to invoke the API using CURL which is what i based my above code on. (https://docs.kanboard.org/en/latest/api/examples.html)
curl \
-u "jsonrpc:19ffd9709d03ce50675c3a43d1c49c1ac207f4bc45f06c5b2701fbdf8929" \
-d '{"jsonrpc": "2.0", "method": "getAllProjects", "id": 1}' \
http://localhost/kanboard/jsonrpc.php
Updated
You Should use an Hashtable for the headers IDictionary parameter, like this:
Regarding your base64 string, i don't know the ConvertTo-Base64 you are using, anyway this is how i do it:
$kanboardtoken = "jsonrpc:57c9d99c1f5629959d237822a000a20b22913c71688e6520e5638beec11b"
$Base64 = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($kanboardtoken))
$headers = #{
'X-API-Auth' = $Base64
}
$payload = #{ jsonrpc = "2.0";
method = "getAllProjects";
id = 1;
} | ConvertTo-Json
$responseKan = Invoke-RestMethod -Uri $kanboardserver -Method Post -Headers $headers -Body $payload -ErrorAction Stop
Thanks, So i'm back to an error I have come across before, which is an authorization problem.
Invoke-RestMethod : {"jsonrpc":"2.0","error":{"code":401,"message":"Unauthorized"},"id":null}
I know the token is correct as I have mocked up a php client and the API is working.
The latest code that gives me this error is below, it has to be with the way the header is formed, I can't think of anything else.
$kanboardserver = "http://mykanboardserver/jsonrpc.php"
$kanboardtoken = "jsonrpc:57c9d99c1f5629959d237822a000a20b22913c71688e6520e5638beec11b" | ConvertTo-Base64
$headers = #{
'X-API-Auth' = $kanboardtoken
}
$payload = #{ jsonrpc = "2.0";
method = "getAllProjects";
id = 1;
} | ConvertTo-Json
$responseKan = Invoke-RestMethod -Uri $kanboardserver -Method Post -Headers $headers -Body $payload -ErrorAction Stop
Write-Host $responseKan