Powershell script for azure function to post to a log analytics - powershell

I am trying to write an azure function which posts data to a specific table e.g. (Table_CL) inside log analytics workspace. I am running into errors.
Install-Module -Name Az.OperationalInsights -Scope CurrentUser -Force
$logAnalyticsClient = Get-AzOperationalInsightsWorkspace -ResourceGroupName "rg-name" -Name "log-analytics-wsp"
$WorkspaceId = "/subscriptions/subscriptionID/resourceGroups/rg-name/providers/Microsoft.OperationalInsights/workspaces/log-analytics-wsp"
$SharedKey = "some-value"
$CustomTableName = "Table_CL"
# Obtain an authentication token
$tenantId = 'some value'
$clientId = 'value of Function App identity client IOD'
$Uri = "https://log-analytics-wsp.ods.opinsights.azure.com/api/logs?api-version=2016-04-01"
$Body = #{
"EventId" = $eventGridEvent.id;
"eventType" = $eventGridEvent.eventType;
"subject" = $eventGridEvent.subject;
"TimeGenerated" = [datetime]$eventGridEvent.eventTime;
"data" = (ConvertTo-Json -InputObject $eventGridEvent.data);
"dataVersion" = $eventGridEventvent.dataVersion;
"metadataVersion" = $eventGridEvent.metadataVersion;
}
$signature = $SharedKey + (get-date -uformat '%a, %d %b %Y %H:%M:%S ')
$hex = ""
foreach ($byte in $signature) {
$hex += "{0:x2}" -f $byte
}
$Signature = $hex
$Headers = #{
"Content-Type" = "application/json"
"Log-Type" = $CustomTableName
"Authorization" = $Signature
}
# Use the authentication token to send a request to the Log Analytics API
Invoke-WebRequest -Uri $Uri -Method Post -Body $Body -Headers $Headers
I am getting errors, can someone please guide me on how to post to specific table in log analytics using powershell script.
I am getting the following errors.
2023-01-18T19:56:48Z [Error] ERROR: The format of value 'some-shared-key-value-==Wed, 18 Jan 2023 19:56:47 GMT' is invalid.
Exception :
Type : System.FormatException
TargetSite :
Name : ParseValue
DeclaringType : System.Net.Http.Headers.HttpHeaderParser, System.Net.Http, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
MemberType : Method
Module : System.Net.Http.dll
Message : The format of value 'some-shared-key-value-Wed, 18 Jan 2023 19:56:47 GMT' is invalid.
Source : System.Net.Http
HResult : -2146233033
StackTrace :
at System.Net.Http.Headers.HttpHeaderParser.ParseValue(String value, Object storeValue, Int32& index)
at System.Net.Http.Headers.HttpHeaders.ParseAndAddValue(HeaderDescriptor descriptor, HeaderStoreItemInfo info, String value)
at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.GetRequest(Uri uri)
at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.ProcessRecord()
at System.Management.Automation.Cmdlet.DoProcessRecord()
at System.Management.Automation.CommandProcessor.ProcessRecord()
CategoryInfo : NotSpecified: (:) [Invoke-WebRequest], FormatException
FullyQualifiedErrorId : System.FormatException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
InvocationInfo :
MyCommand : Invoke-WebRequest
ScriptLineNumber : 55
OffsetInLine : 1
HistoryId : 1
ScriptName : C:\home\site\wwwroot\EventGridTrigger1\run.ps1
Line : Invoke-WebRequest -Uri $Uri -Method Post -Body $Body -Headers $Headers
PositionMessage : At C:\home\site\wwwroot\EventGridTrigger1\run.ps1:55 char:1
+ Invoke-WebRequest -Uri $Uri -Method Post -Body $Body -Headers $Header …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PSScriptRoot : C:\home\site\wwwroot\EventGridTrigger1
PSCommandPath : C:\home\site\wwwroot\EventGridTrigger1\run.ps1
InvocationName : Invoke-WebRequest
CommandOrigin : Internal
ScriptStackTrace : at <ScriptBlock>, C:\home\site\wwwroot\EventGridTrigger1\run.ps1: line 55
Any suggestions? I am looking for way to post to Log Analytics workspace specific table, any help would be great.
Thanks

I had to use the following API from azure docs. This works neatly as it is. Just plug the values and the post command is able to create table in Azure Monitor.
Following is the reference for the docs as well.
# Replace with your Workspace ID
$CustomerId = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
# Replace with your Primary Key
$SharedKey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
# Specify the name of the record type that you'll be creating
$LogType = "MyRecordType"
# Optional name of a field that includes the timestamp for the data. If the time field is not specified, Azure Monitor assumes the time is the message ingestion time
$TimeStampField = ""
# Create two records with the same set of properties to create
$json = #"
[{ "StringValue": "MyString1",
"NumberValue": 42,
"BooleanValue": true,
"DateValue": "2019-09-12T20:00:00.625Z",
"GUIDValue": "9909ED01-A74C-4874-8ABF-D2678E3AE23D"
},
{ "StringValue": "MyString2",
"NumberValue": 43,
"BooleanValue": false,
"DateValue": "2019-09-12T20:00:00.625Z",
"GUIDValue": "8809ED01-A74C-4874-8ABF-D2678E3AE23D"
}]
"#
# Create the function to create the authorization signature
Function Build-Signature ($customerId, $sharedKey, $date, $contentLength, $method, $contentType, $resource)
{
$xHeaders = "x-ms-date:" + $date
$stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource
$bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash)
$keyBytes = [Convert]::FromBase64String($sharedKey)
$sha256 = New-Object System.Security.Cryptography.HMACSHA256
$sha256.Key = $keyBytes
$calculatedHash = $sha256.ComputeHash($bytesToHash)
$encodedHash = [Convert]::ToBase64String($calculatedHash)
$authorization = 'SharedKey {0}:{1}' -f $customerId,$encodedHash
return $authorization
}
# Create the function to create and post the request
Function Post-LogAnalyticsData($customerId, $sharedKey, $body, $logType)
{
$method = "POST"
$contentType = "application/json"
$resource = "/api/logs"
$rfc1123date = [DateTime]::UtcNow.ToString("r")
$contentLength = $body.Length
$signature = Build-Signature `
-customerId $customerId `
-sharedKey $sharedKey `
-date $rfc1123date `
-contentLength $contentLength `
-method $method `
-contentType $contentType `
-resource $resource
$uri = "https://" + $customerId + ".ods.opinsights.azure.com" + $resource + "?api-version=2016-04-01"
$headers = #{
"Authorization" = $signature;
"Log-Type" = $logType;
"x-ms-date" = $rfc1123date;
"time-generated-field" = $TimeStampField;
}
$response = Invoke-WebRequest -Uri $uri -Method $method -ContentType $contentType -Headers $headers -Body $body -UseBasicParsing
return $response.StatusCode
}
# Submit the data to the API endpoint
Post-LogAnalyticsData -customerId $customerId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($json)) -logType $logType

Related

Azure Function: Writing into ADLS Gen2 with PowerShell Script inside Azure Function App

I am newbie to Azure Functions. So I'm trying to write into ADLS Gen2 Storage Account Containers with the help of writing a PowerShell script inside Azure Function in Azure Function APP with Elastic Premium Plan.
Here, I am able to write the Data File to Storage Account but there is "No Data" in that file.
I'm attaching the Code of PowerShell Script written inside the Azure Function along with the Error File after execution.
Experts help me with this. Thanks in Advance.
Code
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Connect to Azure Account
# $username = "warehouse#rentpath.onmicrosoft.com"
# $password = Get-Content D:\PS\passwords\password.txt | ConvertTo-SecureString -Key (Get-Content D:\PS\passwords\aes.key)
# $credential = New-Object System.Management.Automation.PsCredential($username,$password)
# Connect-AzAccount -Credential $userCredential
$username = "warehouse#rentpath.onmicrosoft.com"
$pass = ConvertTo-SecureString "**********" -AsPlainText -Force
$cred = New-Object PSCredential($username,$pass)
Connect-AzAccount -Credential $cred
# Input Variables
$dataFactoryName="dna-production-gen2"
$resourceGroupName="DataLake-Gen2"
# get dataFactory triggers
$triggers=Get-AzDataFactoryV2Trigger -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName
$datas=#()
foreach ($trigger in $triggers) {
# get the trigger run history
$today = Get-Date
$yesterday = $today.AddDays(-1)
$splat = #{
ResourceGroupName = $trigger.ResourceGroupName
DataFactoryName = $trigger.DataFactoryName
TriggerName = $trigger.Name
TriggerRunStartedAfter = $yesterday
TriggerRunStartedBefore = $today
}
$historys =Get-AzDataFactoryV2TriggerRun #splat
if($historys -ne $null){
# create date
foreach($history in $historys){
$obj =[PsCustomObject]#{
'TriggerRunTimestamp ' = $history.TriggerRunTimestamp
'ResourceGroupName ' =$history.ResourceGroupName
'DataFactoryName' =$history.DataFactoryName
'TriggerName ' = $history.TriggerName
'TriggerRunId'= $history.TriggerRunId
'TriggerType'=$history.TriggerType
'Status' =$history.Status
}
# add data to an array
$datas += $obj
}
}
}
# convert data to csv string
$contents =(($datas | ConvertTo-Csv -NoTypeInformation) -join [Environment]::NewLine)
# upload to Azure Data Lake Store Gen2
#1. Create a sas token
$accountName="dna2020gen2"
# $path = New-Item -ItemType Directory -Path ".\$((Get-Date).ToString('yyyy-MM-dd'))"
$YY = (Get-Date).year
$MM = (Get-Date).month
$DD = get-date –f dd
$fileSystemName="dev"
$filePath="Input/Triggers/YYYY=$YY/MM=$MM/DD=$DD/data.csv"
$account = Get-AzStorageAccount -ResourceGroupName 'DataLake-Gen2' -Name $accountName
$sas= New-AzStorageAccountSASToken -Service Blob -ResourceType Service,Container,Object `
-Permission "racwdlup" -StartTime (Get-Date).AddMinutes(-10) `
-ExpiryTime (Get-Date).AddHours(2) -Context $account.Context
$baseUrl ="https://{0}.dfs.core.windows.net/{1}/{2}{3}" -f $accountName , $fileSystemName, $filePath, $sas
#2. Create file
$endpoint =$baseUrl +"&resource=file"
Invoke-RestMethod -Method Put -Uri $endpoint -Headers #{"Content-Length" = 0} -UseBasicParsing
$r = Invoke-WebRequest -Uri https://management.azure.com/subscriptions/{guid}/resourcegroups?api-version=2016-09-01 -Method GET -Headers $authHeaders
$r.Headers["x-ms-ratelimit-remaining-subscription-reads"]
#3 append data
$endpoint =$baseUrl +"&action=append&position=0"
Invoke-RestMethod -Method Patch -Uri $endpoint -Headers #{"Content-Length" = $contents.Length} -Body $contents -UseBasicParsing
#4 flush data
#$endpoint =$baseUrl + ("&action=flush&position={0}" -f $contents.Length)
#Invoke-RestMethod -Method Patch -Uri $endpoint -UseBasicParsing
#Check the result (get data)
Invoke-RestMethod -Method Get -Uri $baseUrl -UseBasicParsing
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $contents
})
Error File:
Connected!
2021-01-22T06:56:02.230 [Information] Executing 'Functions.HttpTrigger1' (Reason='This function was programmatically called via the host APIs.', Id=795e78c8-d6d7-4fcc-b4c0-115951b27807)
2021-01-22T06:56:16.039 [Warning] WARNING: TenantId '0c777d2e-f69e-41e4-8dc2-28fcf4c9604b' contains more than one active subscription. First one will be selected for further use. To select another subscription, use Set-AzContext.
2021-01-22T06:56:16.323 [Information] OUTPUT:
2021-01-22T06:56:31.207 [Information] OUTPUT: Account SubscriptionName TenantId Environment
2021-01-22T06:56:31.216 [Information] OUTPUT: ------- ---------------- -------- -----------
2021-01-22T06:56:31.216 [Information] OUTPUT: warehouse#rentpath.onmicrosoft.com Data Analytics 0c777d2e-f69e-41e4-8dc2-28fcf4c9604b AzureCloud
2021-01-22T06:56:31.216 [Information] OUTPUT:
2021-01-22T06:56:31.775 [Error] ERROR: The cmdlet cannot run because the -ContentType parameter is not a valid Content-Type header. Specify a valid Content-Type for -ContentType, then retry. To suppress header validation, supply the -SkipHeaderValidation parameter.Exception :Type : System.Management.Automation.ValidationMetadataExceptionErrorRecord :Exception :Type : System.Management.Automation.ParentContainsErrorRecordExceptionMessage : The cmdlet cannot run because the -ContentType parameter is not a valid Content-Type header. Specify a valid Content-Type for -ContentType, then retry. To suppress header validation, supply the -SkipHeaderValidation parameter.HResult : -2146233087CategoryInfo : MetadataError: (:) [], ParentContainsErrorRecordExceptionFullyQualifiedErrorId : RuntimeExceptionTargetSite :Name : ThrowTerminatingErrorDeclaringType : System.Management.Automation.MshCommandRuntime, System.Management.Automation, Version=7.0.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35MemberType : MethodModule : System.Management.Automation.dllStackTrace :at System.Management.Automation.MshCommandRuntime.ThrowTerminatingError(ErrorRecord errorRecord)Message : The cmdlet cannot run because the -ContentType parameter is not a valid Content-Type header. Specify a valid Content-Type for -ContentType, then retry. To suppress header validation, supply the -SkipHeaderValidation parameter.InnerException :Type : System.FormatExceptionTargetSite :Name : ParseAndAddValueDeclaringType : System.Net.Http.Headers.HttpHeadersMemberType : MethodModule : System.Net.Http.dllStackTrace :at System.Net.Http.Headers.HttpHeaders.ParseAndAddValue(HeaderDescriptor descriptor, HeaderStoreItemInfo info, String value)at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.FillRequestStream(HttpRequestMessage request)Message : Cannot add value because header 'Content-Length' does not support multiple values.Source : System.Net.HttpHResult : -2146233033Source : System.Management.AutomationHResult : -2146233087CategoryInfo : InvalidArgument: (:) [Invoke-RestMethod], ValidationMetadataExceptionFullyQualifiedErrorId : WebCmdletContentTypeException,Microsoft.PowerShell.Commands.InvokeRestMethodCommandInvocationInfo :MyCommand : Invoke-RestMethodScriptLineNumber : 92OffsetInLine : 1HistoryId : 1ScriptName : C:\home\site\wwwroot\HttpTrigger1\run.ps1Line : Invoke-RestMethod -Method Patch -Uri $endpoint -Headers #{"Content-Length" = $contents.Length} -Body $contents -UseBasicParsingPositionMessage : At C:\home\site\wwwroot\HttpTrigger1\run.ps1:92 char:1+ Invoke-RestMethod -Method Patch -Uri $endpoint -Headers #{"Content-Le …+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~PSScriptRoot : C:\home\site\wwwroot\HttpTrigger1PSCommandPath : C:\home\site\wwwroot\HttpTrigger1\run.ps1InvocationName : Invoke-RestMethodCommandOrigin : InternalScriptStackTrace : at <ScriptBlock>, C:\home\site\wwwroot\HttpTrigger1\run.ps1: line 92Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: ERROR: The cmdlet cannot run because the -ContentType parameter is not a valid Content-Type header. Specify a valid Content-Type for -ContentType, then retry. To suppress header validation, supply the -SkipHeaderValidation parameter.Exception :Type : System.Management.Automation.ValidationMetadataExceptionErrorRecord :Exception :Type : System.Management.Automation.ParentContainsErrorRecordExceptionMessage : The cmdlet cannot run because the -ContentType parameter is not a valid Content-Type header. Specify a valid Content-Type for -ContentType, then retry. To suppress header validation, supply the -SkipHeaderValidation parameter.HResult : -2146233087CategoryInfo : MetadataError: (:) [], ParentContainsErrorRecordExceptionFullyQualifiedErrorId : RuntimeExceptionTargetSite :Name : ThrowTerminatingErrorDeclaringType : System.Management.Automation.MshCommandRuntime, System.Management.Automation, Version=7.0.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35MemberType : MethodModule : System.Management.Automation.dllStackTrace :at System.Management.Automation.MshCommandRuntime.ThrowTerminatingError(ErrorRecord errorRecord)Message : The cmdlet cannot run because the -ContentType parameter is not a valid Content-Type header. Specify a valid Content-Type for -ContentType, then retry. To suppress header validation, supply the -SkipHeaderValidation parameter.InnerException :Type : System.FormatExceptionTargetSite :Name : ParseAndAddValueDeclaringType : System.Net.Http.Headers.HttpHeadersMemberType : MethodModule : System.Net.Http.dllStackTrace :at System.Net.Http.Headers.HttpHeaders.ParseAndAddValue(HeaderDescriptor descriptor, HeaderStoreItemInfo info, String value)at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.FillRequestStream(HttpRequestMessage request)Message : Cannot add value because header 'Content-Length' does not support multiple values.Source : System.Net.HttpHResult : -2146233033Source : System.Management.AutomationHResult : -2146233087CategoryInfo : InvalidArgument: (:) [Invoke-RestMethod], ValidationMetadataExceptionFullyQualifiedErrorId : WebCmdletContentTypeException,Microsoft.PowerShell.Commands.InvokeRestMethodCommandInvocationInfo :MyCommand : Invoke-RestMethodScriptLineNumber : 92OffsetInLine : 1HistoryId : 1ScriptName : C:\home\site\wwwroot\HttpTrigger1\run.ps1Line : Invoke-RestMethod -Method Patch -Uri $endpoint -Headers #{"Content-Length" = $contents.Length} -Body $contents -UseBasicParsingPositionMessage : At C:\home\site\wwwroot\HttpTrigger1\run.ps1:92 char:1+ Invoke-RestMethod -Method Patch -Uri $endpoint -Headers #{"Content-Le …+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~PSScriptRoot : C:\home\site\wwwroot\HttpTrigger1PSCommandPath : C:\home\site\wwwroot\HttpTrigger1\run.ps1InvocationName : Invoke-RestMethodCommandOrigin : InternalScriptStackTrace : at <ScriptBlock>, C:\home\site\wwwroot\HttpTrigger1\run.ps1: line 92Exception: The cmdlet cannot run because the -ContentType parameter is not a valid Content-Type header. Specify a valid Content-Type for -ContentType, then retry. To suppress header validation, supply the -SkipHeaderValidation parameter.Stack: at System.Management.Automation.MshCommandRuntime.ThrowTerminatingError(ErrorRecord errorRecord)
2021-01-22T06:56:31.872 [Information] OUTPUT:
2021-01-22T06:56:31.872 [Information] OUTPUT:
2021-01-22T06:56:31.872 [Information] Executed 'Functions.HttpTrigger1' (Succeeded, Id=795e78c8-d6d7-4fcc-b4c0-115951b27807, Duration=29642ms)
Regarding the issue, please refer to the following script
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
$username = "jimxxxx.onmicrosoft.com"
$pass = ConvertTo-SecureString "xxx" -AsPlainText -Force
$cred = New-Object PSCredential($username,$pass)
Connect-AzAccount -Credential $cred -Tenant "xxx.onmicrosoft.com"
$triggers=Get-AzDataFactoryV2Trigger -ResourceGroupName test001 -DataFactoryName testfactory05
$datas=#()
foreach ($trigger in $triggers) {
# get the trigger run history
$today = Get-Date
$yesterday = $today.AddDays(-1)
$splat = #{
ResourceGroupName = $trigger.ResourceGroupName
DataFactoryName = $trigger.DataFactoryName
TriggerName = $trigger.Name
TriggerRunStartedAfter = $yesterday
TriggerRunStartedBefore = $today
}
$historys =Get-AzDataFactoryV2TriggerRun #splat
if($historys -ne $null){
# create date
foreach($history in $historys){
$obj =[PsCustomObject]#{
'TriggerRunTimestamp ' = $history.TriggerRunTimestamp
'ResourceGroupName ' =$history.ResourceGroupName
'DataFactoryName' =$history.DataFactoryName
'TriggerName ' = $history.TriggerName
'TriggerRunId'= $history.TriggerRunId
'TriggerType'=$history.TriggerType
'Status' =$history.Status
}
# add data to an array
$datas += $obj
}
}
}
$contents =(($datas | ConvertTo-Csv -NoTypeInformation) -join [Environment]::NewLine)
$accountName="testadls05"
$YY = (Get-Date).year
$MM = (Get-Date).month
$DD = get-date –f dd
$fileSystemName="test"
$filePath="Input/Triggers/YYYY=$YY/MM=$MM/DD=$DD/data.csv"
$account = Get-AzStorageAccount -ResourceGroupName 'andywin7' -Name $accountName
$sas= New-AzStorageAccountSASToken -Service Blob -ResourceType Service,Container,Object `
-Permission "racwdlup" -StartTime (Get-Date).AddMinutes(-10) `
-ExpiryTime (Get-Date).AddHours(2) -Context $account.Context
$baseUrl ="https://{0}.dfs.core.windows.net/{1}/{2}{3}" -f $accountName , $fileSystemName, $filePath, $sas
#2. Create file
Write-Host "Create file"
$endpoint =$baseUrl +"&resource=file"
Invoke-RestMethod -Method Put -Uri $endpoint -Headers #{"Content-Length" = 0} -UseBasicParsing
$currentAzureContext = Get-AzContext
$azureRmProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile;
$profileClient = New-Object Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient($azureRmProfile);
$token=$profileClient.AcquireAccessToken($currentAzureContext.Subscription.TenantId).AccessToken;
$authHeaders=#{"Authorization" ="Bearer $token"}
$r = Invoke-WebRequest -Uri "https://management.azure.com/subscriptions/$($currentAzureContext.Subscription.Id)/resourcegroups?api-version=2016-09-01" -Method GET -Headers $authHeaders
$r.Headers["x-ms-ratelimit-remaining-subscription-reads"]
#3 append data
Write-Host "append data"
$endpoint =$baseUrl +"&action=append&position=0"
$body= [system.Text.Encoding]::UTF8.GetBytes($contents)
Invoke-RestMethod -Method Patch -Uri $endpoint -Body $body -UseBasicParsing
#4 flush data
$endpoint =$baseUrl + ("&action=flush&position={0}" -f $body.Length)
Invoke-RestMethod -Method Patch -Uri $endpoint -UseBasicParsing
#Check the result (get data)
Invoke-RestMethod -Method Get -Uri $baseUrl -UseBasicParsing
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $contents
})

Unable to ingest JSON stream data to Azure Event Hub

I would like to post the JSON results which I am getting from an API endpoint to the Azure Event Hub $default consumer group but I am getting the below error:
Invoke-RestMethod : The remote server returned an error: (401) Unauthorized.
At H:\Users\User1 - scripts\get_mel_streetparking_data.ps1:34 char:1
+ Invoke-RestMethod -Uri $URI -Method $method -Headers $headers -Body $res ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke- RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I wrote the below piece of powershell script where the following piece of get code runs fine:
$url = "https://data.melbourne.vic.gov.au/resource/vh2v-4nfs"
$apptoken = "abcdasdadaaf"
# Set header to accept JSON
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept","application/json")
$headers.Add("X-App-Token",$apptoken)
$results = Invoke-RestMethod -Uri $url -Method get -Headers $headers
but below piece of code fails with the above given error:
$method = "POST"
$URI = "https://[servicebusNamespace].servicebus.windows.net/[eventHubPath]/messages"
$signature = "SharedAccessSignature sr=[servicebusNamespace].servicebus.windows.net%2feventhub- streetparking&sig=%3dgZfDHEGN8lVEGgqu4N64TW70BLuSKARSKgMPeRByc%5d&se=604985&skn=RootManageSharedAccessKe"
# API headers
$headers = #{
"Authorization"=$signature;
"Content-Type"="application/json";
}
# create Request Body
#$body = "$results"
# execute the Azure REST API
Invoke-RestMethod -Uri $URI -Method $method -Headers $headers -Body $results
I generated SAS token with the help of the below code:
[Reflection.Assembly]::LoadWithPartialName("System.Web")| out-null
$URI="[servicebusNamespace].servicebus.windows.net/[eventHubPath]"
$Access_Policy_Name="RootManageSharedAccessKey"
$Access_Policy_Key="Root key value"
#Token expires now+3000
$Expires=([DateTimeOffset]::Now.ToUnixTimeSeconds())+3000
$SignatureString=[System.Web.HttpUtility]::UrlEncode($URI)+ "`n" + [string]$Expires
$HMAC = New-Object System.Security.Cryptography.HMACSHA256
$HMAC.key = [Text.Encoding]::ASCII.GetBytes($Access_Policy_Key)
$Signature = $HMAC.ComputeHash([Text.Encoding]::ASCII.GetBytes($SignatureString))
$Signature = [Convert]::ToBase64String($Signature)
$SASToken = "SharedAccessSignature sr=" + [System.Web.HttpUtility]::UrlEncode($URI) + "&sig=" + [System.Web.HttpUtility]::UrlEncode($Signature) + "&se=" + $Expires + "&skn=" + $Access_Policy_Name
$SASToken
Please help me why am I seeing unauthorized error
It was my bad that I did not replace the below values before the SAS token generation.
$Access_Policy_Name="RootManageSharedAccessKey"
$Access_Policy_Key="Root key value"

PowerShell Slack Function

I am trying to write a PowerShell function to call a Slack webhook. I took the function from a Reddit post, but the function seems to be failing with a parse error. Also I removed the webhook.
Here is the code:
function Send-SlackMessage {
Param (
[Parameter(Mandatory=$true, Position=0)]$Text,
$Url = "https://hooks.slack.com/services/xxxxx",
# Parameters below are optional and will fall back to the default
$Username = "XXXXXXX",
$Channel = "XXXXXXX",
$Emoji = "XXXXXX"
)
$body = #{ text=$Text; channel=$Channel; username=$Username; icon_emoji=$Emoji } | ConvertTo-Json
Invoke-WebRequest -Method Post -Uri $Url -Body $body
}
and the error:
At line:12 char:67
+ ... y = #{ text=$Text; channel=$Channel; username=$Username; icon_emoji=$ ...
+
Missing '=' #{ text=$Text; channel=$Channel; username=$Username; icon_emoji=$ ...
The hash literal was incommplete.
+ CategoryInfo :ParserError (:) [], ParentContainsErrorRecordException
+ FillyQualifiedErrorId : MissingEqualsInHashLiteral
Where's the function name? You are missing a comma after the 2nd param, and have an extra comma after the last param.
function whatever ()
{
param (
[Parameter(Mandatory=$true, Position=0)]$Text,
$Url="https://hooks.slack.com/services/xxxxx",
# Parameters below are optional and will fall back to the default
$Username = "XXXXXXX",
$Channel = "XXXXXXX",
$Emoji = "XXXXXX"
)
$body = #{ text=$Text; channel=$Channel; username=$Username; icon_emoji=$Emoji } | ConvertTo-Json
Invoke-WebRequest -Method Post -Uri $Url -Body $body
}

Powershell Invoke RestMethod error 415 Unsupported Media Type

I'm trying to execute Azure Table Storage API using Powershell Invoke-RestMethod but it returns 415 error.
Here's Powershell code:
$accessKey = "accesskey"
$account_name = "storage account"
$table_name = "table storage"
$date = "Fri, 10 Nov 2017 00:47:55 GMT"
$api_version = "2015-12-11"
$table_url = "https://$account_name.table.core.windows.net/$table_name"
$data_type = "application/json"
$signature = "generated signature"
$body = #{
Address="Mountain View";
Age="23"
}
$json = $body | Convertto-JSON
$table_headers = #{
"x-ms-date" = $date
"x-ms-version" = $api_version
"Authorization" = "SharedKey $account_name`:$signature"
"Content-Type" = $data_type
}
Invoke-RestMethod -Method POST -Uri $table_url -Headers $table_headers -Body $json -Verbose
Remote server retunrned error:
詳細: GET
https://preprodtelemaapim.management.azure-api.net/reports/byApi?api-version=2017-03-01&$filter=timestamp+ge+datetime'2017-10-16T00:00:00'+and+timestamp+le+datetime'2017-10-17T00:00:00'
with 0-byte payload 詳細: received 4841-byte response of content type
application/json; charset=utf-8 詳細: POST
https://telemaapimlog.table.core.windows.net/reporttable with -1-byte
payload Invoke-RestMethod : リモート サーバーがエラーを返しました*: (415) Unsupported
Media Type 発生場所 C:\Users\sasaki.hikaru\Desktop\repot.ps1:83 文字:1
+ Invoke-RestMethod -Method POST -Uri $table_url -Headers $table_headers -Body $js ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest)
[Invoke-RestMethod]、WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I have read some similar problems in this site but they are no content-type or json description issues mostly so I assume that there are no problems in my code.
Anybody have ideas? Please help me.
Interesting problem. The reason you're getting this error is because you have not specified the format of response data i.e value for Accept request header. Because you have not specified this value, storage service treats its value as XML which is not supported for the storage service version that you have specified.
Once you include this Accept header (and specify value for PartitionKey and RowKey in your $body), things should work just fine.
Here's the code that I wrote:
$accessKey = "account key"
$account_name = "account name"
$table_name = "table name"
$date = ([System.DateTime]::Now).ToString("R")
$api_version = "2016-05-31"
$table_url = "https://$account_name.table.core.windows.net/$table_name"
$data_type = "application/json"
$canonicalResource = "/$account_name/$table_name";
$stringToSign = "POST`n`n$data_type`n$date`n$canonicalResource";
$utf8enc = [System.Text.Encoding]::UTF8;
$bytes = $utf8enc.GetBytes($stringToSign)
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [System.Convert]::FromBase64String($accessKey)
$signature = [System.Convert]::ToBase64String($hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($stringToSign)))
$body = #{
PartitionKey = "1";
RowKey = [System.guid]::NewGuid().ToString();
Address="Mountain View";
Age="23";
}
$json = $body | Convertto-JSON
$table_headers = #{
"x-ms-date" = $date
"x-ms-version" = $api_version
"Authorization" = "SharedKey $account_name`:$signature"
"Content-Type" = $data_type
"Accept" = "application/json;odata=fullmetadata"
}
Invoke-RestMethod -Method POST -Uri $table_url -Headers $table_headers -Body $json -Verbose

Getting 404 on Powershell Query of Cosmos DB

I'm using this script from the download in this link.
https://gallery.technet.microsoft.com/scriptcenter/How-to-query-Azure-Cosmos-0a9aa517
However for some reason I am getting a 404 response.
I copy-pasted the url of the db directly. Putting in a fake URL gives me a "could not be resolved" error so I know the location exists.
Based on the Azure CosmosDB API documentation here:
https://learn.microsoft.com/en-us/rest/api/documentdb/databases
the $databaseID is user-set and just has to be unique, so I've set it to be the same as the db name and assigned that to the url.
Changing it to be different still gives me the same 404 response message (below).
Edit: Removed original commenting intro for readability
Powershell Script:
# add necessary assembly
#
Add-Type -AssemblyName System.Web
# generate authorization key
Function Generate-MasterKeyAuthorizationSignature
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$verb,
[Parameter(Mandatory=$true)][String]$resourceLink,
[Parameter(Mandatory=$true)][String]$resourceType,
[Parameter(Mandatory=$true)][String]$dateTime,
[Parameter(Mandatory=$true)][String]$key,
[Parameter(Mandatory=$true)][String]$keyType,
[Parameter(Mandatory=$true)][String]$tokenVersion
)
$hmacSha256 = New-Object System.Security.Cryptography.HMACSHA256
$hmacSha256.Key = [System.Convert]::FromBase64String($key)
$payLoad = "$($verb.ToLowerInvariant())`n$($resourceType.ToLowerInvariant())`n$resourceLink`n$($dateTime.ToLowerInvariant())`n`n"
$hashPayLoad = $hmacSha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($payLoad))
$signature = [System.Convert]::ToBase64String($hashPayLoad);
[System.Web.HttpUtility]::UrlEncode("type=$keyType&ver=$tokenVersion&sig=$signature")
}
# query
Function Query-CosmosDb
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$EndPoint,
[Parameter(Mandatory=$true)][String]$DataBaseId,
[Parameter(Mandatory=$true)][String]$CollectionId,
[Parameter(Mandatory=$true)][String]$MasterKey,
[Parameter(Mandatory=$true)][String]$Query
)
$Verb = "POST"
$ResourceType = "docs";
$ResourceLink = "dbs/$DatabaseId/colls/$CollectionId"
$dateTime = [DateTime]::UtcNow.ToString("r")
$authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
$queryJson = #{query=$Query} | ConvertTo-Json
$header = #{authorization=$authHeader;"x-ms-documentdb-isquery"="True";"x-ms-version"="2017-02-22";"x-ms-date"=$dateTime}
$contentType= "application/json "# The original said "application/query+json", I tried both
$queryUri = "$EndPoint$ResourceLink/docs"
$result = Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header -Body $queryJson
$result | ConvertTo-Json -Depth 10
}
# fill the target cosmos database endpoint uri, database id, collection id and masterkey
$DatabaseName = "" # name goes here
$MasterKey = "" #key goes here
$CollectionId = "transientUsers"
$DatabaseId = $DatabaseName
$CosmosDBEndPoint = "https://$DatabaseId.documents.azure.com:443/"
# query string
$Query = "SELECT * FROM transientUsers"
# execute
Query-CosmosDb -EndPoint $CosmosDBEndPoint -DataBaseId $DataBaseId -CollectionId $CollectionId -MasterKey $MasterKey -Query $Query
Error I'm getting:
Invoke-RestMethod : The remote server returned an error: (404) Not Found.
At D:\querycosmos\PowerShell\QueryCosmosDB.ps1:69 char:12
+ ... $result = Invoke-RestMethod -Method $Verb -ContentType $contentType ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I noticed that you used $DatabaseId in two places:
$ResourceLink = "dbs/$DatabaseId/colls/$CollectionId"
and
$CosmosDBEndPoint = "https://$DatabaseId.documents.azure.com:443/"
If $DatabaseId refers to your account name, then you would need to change your $ResourceLink variable and use the name of the database inside your account containing the collection. If however $DatabaseId refers to the database name, then you would need to change $CosmosDBEndPoint and use account name there.