I am using Powershell script to upload data every minute into streaming dataset in PowerBI(It's scheduled with Windows Task Scheduler).
I'm struggling with strange error, that appeared without changing anything.
This is my code:
function invokeRest()
{
$endpoint = "https://api.powerbi.com/beta/..."
$query = #(Invoke-Sqlcmd -Query "select * from LS.dbo.live_Holdbacks;" -ServerInstance $env:computername)
$RetArray = #()
Foreach ($row in $query) {
$payload = #{
"id" = $row.LiveHoldbackID
"RefreshDate" = $row.RefreshDate.ToString("HH:mm")
"CreatedBy" = $row.CreatedBy
"Campaign Code" = $row.CampaignCode
"Campaign Description" = $row.CampaignDescription
"Comment" = $row.Comment
"Customer Number" = $row.CustomerNumber
"Country" = $row.Country
"holdback" = $row.holdback
"upselling" = $row.upselling
}
$RetArray += $payload
}
Write-Host (ConvertTo-Json #($RetArray));
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json #($RetArray)) -Verbose
}
function ClearDataset([string]$authToken)
{
Clear-PBITableRows -authToken $authToken -dataSetName "DEV_live_holdbacks_no_History_1min" -tableName "RealTimeData" -Verbose
}
Import-Module -Name PowerBIPS
$authToken = Get-PBIAuthToken -ClientId "...." -Credential (new-object System.Management.Automation.PSCredential("......",(ConvertTo-SecureString -String "...." -AsPlainText -Force)))
$group = Get-PBIGroup -authToken $authToken -name "..."
Set-PBIGroup -id $group.id
$dataSets = Get-PBIDataSet -authToken $authToken -name "DEV_live_holdbacks_no_History_1min" -includeTables -Verbose
#ClearDataset $authToken
try {
ClearDataset $authToken
invokeRest
}
catch {
Write-Host 'Sth goes bad'
Write-Host $_
# do something with $_, log it, more likely
}
API info is taken directly from PowerBI, every URL is ok, and everything that I got in console is this:
The remote server returned an error: (400) Bad Request.
It just suddenly stopped working... Anyone got idea how to deal with it?
I figured this out! The problem was related to one of fields - it's type was set to Number and there was string passed into it.
Related
I am trying to write a PowerShell script that allows me to update all the names of our devices in Intune [430ish devices] to reflect our asset tags. When they were imported into our tenant, they were given the serialNumber of the device as their deviceName. All permissions for the API have been applied:
API Permissions:
Device Read
Device Read all
DeviceManagementApps.ReadAll
DeviceManagementApps.ReadWriteAll
DeviceManagementConfiguration.ReadAll
DeviceManagementConfiguration.ReadWriteAll
DeviceManagementManagedDevices.PrivilegedOperations.All
DeviceManagementManagedDevices.ReadAll
DeviceManagementManagedDevices.ReadWriteAll
DeviceManagementRBAC.ReadAll
DeviceManagementRBAC.ReadWriteALL
DeviceManagementServiceConfig.ReadAll
DeviceManagementServiceConfig.ReadWriteAll
User Read
This is the code as far as I can get it, but I am still getting the following error [I apologise for ugly or poorly formatted code, I have had no formal training, all learnt using google-fu!]:
# Setting variables for connecting to the MS API
$ApplicationID = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
$TenantDomainName = "contoso.com"
$AccessSecret = Read-Host "Enter Secret"
# Connect to MSGraph command to run
Connect-MSGraph
# Setting the body of the json
$Body = #{
Grant_Type = "client_credentials"
Scope = "https://graph.microsoft.com/.default"
client_Id = $ApplicationID
Client_Secret = $AccessSecret
}
# Authenticating the connection to MSGraph
$ConnectGraph = Invoke-RestMethod -Uri "https://login.microsoftonline.com/$TenantDomainName/oauth2/v2.0/token" `
-Method POST -Body $Body
$token = $ConnectGraph.access_token
# Importing the CSV of device information
$csvfile = "C:\<Path to file>"
Import-Csv $csvfile | ForEach-Object {
$serialNumber = $_.serialNumber;
$tag = $_.tag;
$deviceId = $serialNumber
Write-Host "Renaming machine from: $deviceID to: $tag" -ForegroundColor Cyan
# Getting the Device from the CSV and then putting it into MSGraph compatible Json
$DeviceToRename = Get-IntuneManagedDevice -Filter ("serialNumber eq '$serialNumber'")
Foreach ($Device in $DeviceToRename) {
$Resource = "deviceManagement/managedDevices('$DeviceId')/setDeviceName"
$graphApiVersion = "Beta"
$uri = "https://graph.microsoft.com/beta/deviceManagement/managedDevices/executeAction"
#This JSON format doesnt work
# $JSONPayload = #"
# { <NEW>
# "body": <NEW>
# {
# action: "setDeviceName",
# actionName: "setDeviceName",
# deviceName: "$tag",
# realaction: "setDeviceName",
# restartNow: false
# }
# } <NEW>
#"#
#Don't know if this works properly either?
$JSONPayload = #"
{
"#odata.type": "#microsoft.graph.managedDevice",
"actionName": "setDeviceName",
"deviceName": "$tag"
}
"#
# Writing out to check if this is working correctly
Write-Host $JSONPayload
# Converting $JSONPayload to an actual workable JSON
$convertedJSON = ConvertTo-Json $JSONPayload
try {
Invoke-MSGraphRequest -Url $uri -HttpMethod PATCH -Body $JSONPayload -ContentType "application/Json" -Verbose
} catch {
# Dig into the exception to get the Response details.
Write-Host "StatusCode:" "$_.Exception.Response.StatusCode.value__"
Write-Host "StatusDescription:" "$_.Exception.Response.StatusDescription"
Write-Host "StatusCode2:" "$_.ErrorDetails.Message"
}
}
}
Error response:
StatusCode: A parameter cannot be found that matches parameter name 'Body'..Exception.Response.StatusCode.value__
StatusDescription: A parameter cannot be found that matches parameter name 'Body'..Exception.Response.StatusDescription
StatusCode2: A parameter cannot be found that matches parameter name 'Body'..ErrorDetails.Message
Thanks
Tom
I had similar problems some months ago manipulating intune devices from an powershell runbook over graph. In my case the json body was the problem. I had to define the body first as hashtable and then convert it to json. Try something like this:
# JSONPayload as hashtable instead of string
$JSONPayload = #{
"#odata.type" = "#microsoft.graph.managedDevice"
"actionName" = "setDeviceName"
"deviceName" = "$tag"
}
# Writing out to check if this is working correctly
$JSONPayload
# Converting $JSONPayload to an actual workable JSON
$convertedJSON = $JSONPayload | ConvertTo-Json
And then pass the $convertedJSON to your graph call as body:
Invoke-MSGraphRequest -Url $uri -HttpMethod POST -Content $convertedJSON -Verbose
EDIT:
You are calling the endpoint /deviceManagement/managedDevices/executeAction with the http method PATCH. According to this ms docs article you have to call the endpoint with the http method POST.
So here is my code to get the AuthToken for my Tenant ID, this is from Microsoft and generates a JWT to use as authorization in the HTTP header:
function Get-AuthToken {
[cmdletbinding()]
param
(
[Parameter(Mandatory=$true)]
$User
)
$userUpn = New-Object "System.Net.Mail.MailAddress" -ArgumentList $User
$tenant = $userUpn.Host
Write-Host "Checking for AzureAD module..."
$AadModule = Get-Module -Name "AzureAD" -ListAvailable
if ($AadModule -eq $null) {
Write-Host "AzureAD PowerShell module not found, looking for AzureADPreview"
$AadModule = Get-Module -Name "AzureADPreview" -ListAvailable
}
if ($AadModule -eq $null) {
write-host
write-host "AzureAD Powershell module not installed..." -f Red
write-host "Install by running 'Install-Module AzureAD' or 'Install-Module AzureADPreview' from an elevated PowerShell prompt" -f Yellow
write-host "Script can't continue..." -f Red
write-host
exit
}
if($AadModule.count -gt 1){
$Latest_Version = ($AadModule | select version | Sort-Object)[-1]
$aadModule = $AadModule | ? { $_.version -eq $Latest_Version.version }
# Checking if there are multiple versions of the same module found
if($AadModule.count -gt 1){
$aadModule = $AadModule | select -Unique
}
$adal = Join-Path $AadModule.ModuleBase "Microsoft.IdentityModel.Clients.ActiveDirectory.dll"
$adalforms = Join-Path $AadModule.ModuleBase "Microsoft.IdentityModel.Clients.ActiveDirectory.Platform.dll"
}else {
$adal = Join-Path $AadModule.ModuleBase "Microsoft.IdentityModel.Clients.ActiveDirectory.dll"
$adalforms = Join-Path $AadModule.ModuleBase "Microsoft.IdentityModel.Clients.ActiveDirectory.Platform.dll"
}
[System.Reflection.Assembly]::LoadFrom($adal) | Out-Null
[System.Reflection.Assembly]::LoadFrom($adalforms) | Out-Null
$clientId = "d1ddf0e4-d672-4dae-b554-9d5bdfd93547"
$redirectUri = "urn:ietf:wg:oauth:2.0:oob"
$resourceAppIdURI = "https://graph.microsoft.com"
$authority = "https://login.microsoftonline.com/$Tenant"
try {
$authContext = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext" -ArgumentList $authority
$platformParameters = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.PlatformParameters" -ArgumentList "Auto"
$userId = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.UserIdentifier" -ArgumentList ($User, "OptionalDisplayableId")
$authResult = $authContext.AcquireTokenAsync($resourceAppIdURI,$clientId,$redirectUri,$platformParameters,$userId).Result
# If the accesstoken is valid then create the authentication header
if($authResult.AccessToken){
# Creating header for Authorization token, we dont necessarily need it, just the JWT auth token.
$authHeader = #{
'Content-Type'='application/json'
'Authorization'="Bearer " + $authResult.AccessToken
'ExpiresOn'=$authResult.ExpiresOn
}
# Write-Output
return $authResult
# return $authHeader
}
}catch {
write-host $_.Exception.Message -f Red
break
}
}
So basically at the bottom I use this JWT auth token from the function above and place it as an Authorization field in the HTTP header and it should return JSON from the Graph API:
$authData = Get-AuthToken -User acct#pennitout.com
$accessJWToken = $authData.AccessToken
$apiUrl = "https://graph.microsoft.com/v1.0/users?$select=displayName"
Invoke-RestMethod -Headers #{"Authorization" = "Bearer $accessJWToken"} -Uri $apiUrl -Method Get -ContentType "application/json"
And the above code beautifully absolutely uselessly output it returns instead is:
#odata.context #odata.nextLink
-------------- ---------------
https://graph.microsoft.com/v1.0/$metadata#users https://graph.microsoft.com/v1.0/users?=displayName&$skiptoken=RFNwdAIAAQAAABg6YWdyYW50QHRlcnJhbmV1dHJhbC5jb20pVXNlcl85MzA2OWJlYy0zZjFjLTRiNDQtOTZjMS
Please Help with this thanks I really appreciate
Can you please check there might be issue that Token has expired, Please try to generate new token and check it for the Odata next link which might have caused the error.
Here is the document for reference:Token Duration
I am trying to convert the below code to a PowerShell function as I will be using it multiple times in other scripts. I am trying to figure out how I can allow the user to simply pass a SQL Query (a String) to this script that I will turn into a function and have my hash table $row define the keys dynamically based on the column headers and then assign the values when multiple rows are returned by my SQL Query variable $QueryResults. Also, I will have two common variables in each hash table, which are "DateTime" and "ServerName" so those will have to be added as well.
$endpoint = "myPowerBIEndpoint"
$Query = "
myQueryThatReturnsMultipleRecords
"
$GetDate = (Get-Date).AddMinutes(-1)
while($true)
{
if((Get-Date) -gt $GetDate)
{
#Only run the below every 5 minutes
$GetDate = (Get-Date).AddMinutes(5)
#Get list of server to iterate through
$ServerList = Invoke-DbaQuery -SqlInstance "" -Database "" -Query "
EXEC [dbo].[myProc] #MonitorType" -SqlParameters #{MonitorType = "AGHealth"}
}
$ServerResults = #() # Results from each of the servers will be stored here
foreach ($Server in $ServerList)
{
$QueryResults = Invoke-DbaQuery -SqlInstance $ServerList.Cname -Database "" -Query $Query
$DateTime = Get-Date -DisplayHint Datetime -Format "MM/dd/yyyy HH:mm:ss"
foreach ($object in $QueryResults)
{
Write-Host 'Building Payload for' $object.replica_server_name
$row = #{
"DateTime" = $DateTime
"Replica_Server" = $object.replica_server_name
"Log_Send_Queue_Size_mb" = $object.total_log_send_queue_size_mb
"Redo_Queue_Size_mb" = $object.total_redo_queue_size_mb
"Sync_Health" = $object.synchronization_health
"Is_Primary_Replica" = $object.is_primary_replica
"AG_Name" = $object.ag_name
"ServerName" = $Server.Cname
}
$ServerResults += $row
}
}
$payload = #{ "rows" = $ServerResults }
Write-Host 'Invoking Rest Method'
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json $payload)
(ConvertTo-Json $payload)
sleep 60
}
As Invoke-DbaQueryis somehow a wrapper of Invoke-SqlCmd, we can use the schema information stored in the returned PSObject [DataRow] for each row. Each one has a Table member filled by the command describing the resulting schema for the row. In this [DataTable]Table member, we have the [DataColumnCollection]Columns which describes each column of the row as [DataColmun]. We can then use that to feed a hash with the ColumnName member as a key and use the indexed array Item of the row for this ColumnName as value.
The specific corresponding PowerShell code, we add DateTime and ServerName distinctively because they are not coming from the [DataRow] itself :
$row = #{}
$object.Table.Columns | %{ $row.Add($_.ColumnName, $object.Item($_.ColumnName) ) }
$row.Add("DateTime", $DateTime)
$row.Add("ServerName", $Server.Cname)
The resulting script :
$GetDate = (Get-Date).AddMinutes(-1)
while($true)
{
if((Get-Date) -gt $GetDate)
{
#Only run the below every 5 minutes
$GetDate = (Get-Date).AddMinutes(5)
#Get list of server to iterate through
$ServerList = Invoke-DbaQuery -SqlInstance "" -Database "" -Query "
EXEC [dbo].[myProc] #MonitorType" -SqlParameters #{MonitorType = "AGHealth"}
}
$ServerResults = #() # Results from each of the servers will be stored here
foreach ($Server in $ServerList)
{
$QueryResults = Invoke-DbaQuery -SqlInstance $ServerList.Cname -Database "" -Query $Query
$DateTime = Get-Date -DisplayHint Datetime -Format "MM/dd/yyyy HH:mm:ss"
foreach ($object in $QueryResults)
{
Write-Host 'Building Payload for' $object.replica_server_name
$row = #{}
$object.Table.Columns | %{ $row.Add($_.ColumnName, $object.Item($_.ColumnName) ) }
$row.Add("DateTime", $DateTime)
$row.Add("ServerName", $Server.Cname)
$ServerResults += $row
}
}
$payload = #{ "rows" = $ServerResults }
Write-Host 'Invoking Rest Method'
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json $payload)
(ConvertTo-Json $payload)
sleep 60
}
I have the following script that will deploy an Octopus Deploy release based on the parameters I provide it.
param(
[string] $releaseVersion,
[array] $future
)
foreach($line in Get-Content C:\admin\customers.txt) {
$ErrorActionPreference = "Stop";
# Define working variables
$octopusURL = "http://10.2.2.62:8022/api"
$octopusAPIKey = "API-Key"
$headers = #{ "X-Octopus-ApiKey" = $octopusAPIKey }
$spaceName = "Default"
$projectName = "C9-Deployment"
$environmentName = "LabFarm2"
$tenantNames = $line
$date = get-date -Format yyyy-MM-dd
$expiredate = $(date).AddDays(1).ToString("yyyy-MM-dd")
# Get space id
$spaces = Invoke-WebRequest -Uri "$octopusURL/spaces/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$space = $spaces | Where-Object { $_.Name -eq $spaceName }
Write-Host "Using Space named $($space.Name) with id $($space.Id)"
# Get project by name
$projects = Invoke-WebRequest -Uri "$octopusURL/projects/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$project = $projects | Where-Object { $_.Name -eq $projectName }
Write-Host "Using Project named $($project.Name) with id $($project.Id)"
# Create space specific url
$octopusSpaceUrl = "$octopusURL/$($space.Id)"
# Get release by version
$releases = Invoke-RestMethod -Uri "$octopusSpaceUrl/projects/$($project.Id)/releases" -Headers $headers -ErrorVariable octoError
$release = $releases.Items | Where-Object { $_.Version -eq $releaseVersion }
Write-Host "Using Release version $($release.Version) with id $($release.Id)"
# Get environment by name
$environments = Invoke-RestMethod -Uri "$octopusSpaceUrl/environments?partialName=$([uri]::EscapeDataString($environmentName))&skip=0&take=100" -Headers $headers -ErrorVariable octoError
$environment = $environments.Items | Where-Object { $_.Name -eq $environmentName }
Write-Host "Using Environment named $($environment.Name) with id $($environment.Id)"
$tenants = Invoke-WebRequest -Uri "$octopusSpaceUrl/tenants/all" -Headers $headers -ErrorVariable octoError | ConvertFrom-Json
$tenantNames | ForEach-Object {
$name = $_
$tenant = $tenants | Where-Object { $_.Name -eq $name }
if ($future -eq $null) {
write-host "This deployment is for tonight"
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${date}T23:00:00"
QueueTimeExpiry = "${expiredate}T05:00:00"
} | ConvertTo-Json
}
if ($future -ne $null) {
write-host "This deployment will take place on $future"
#Problem Line 64 below
$expirefuturedate = (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${future}T23:00:00"
#problem line 70 below
QueueTimeExpiry = "${expirefuturedate}T05:00:00"
} | ConvertTo-Json
}
Write-Host "Creating deployment with these values: $deploymentBody"
$deployment = Invoke-WebRequest -Uri $octopusSpaceUrl/deployments -Method POST -Headers $headers -Body $deploymentBody -ErrorVariable octoError
}
}
So the problem is on line 64 and 70 where I try to take add one day to the Future parameter. If I run this with only the ReleaseVersion parameter set it will run fine without issues. But if I add a parameter for future like "-Future 2021-03-11" I get the following error:
PS C:\Users\bbelden.CLOUD9\Documents\powershell\Octopus> .\Deploycustom_parm.ps1 -releaseversion 8.1.2103.193 -future 20
21-03-11
Using Space named Default with id Spaces-1
Using Project named C9-Deployment with id Projects-101
Using Release version 8.1.2103.193 with id Releases-12243
Using Environment named LabFarm2 with id Environments-161
This deployment will take place on 2021-03-11
ForEach-Object : Cannot convert 'System.Object[]' to the type 'System.DateTime' required by parameter 'Date'.
Specified method is not supported.
At C:\Users\bbelden.CLOUD9\Documents\powershell\Octopus\Deploycustom_parm.ps1:47 char:16
+ $tenantNames | ForEach-Object {
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [ForEach-Object], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgument,Microsoft.PowerShell.Commands.ForEachObjectCommand
Now if I just remove the line 64 completely and manually set a time date stamp in line 70 like the following:
if ($future -ne $null) {
write-host "This deployment will take place on $future"
#$expirefuturedate = (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
$deploymentBody = #{
ReleaseId = $release.Id
EnvironmentId = $environment.Id
TenantId = $tenant.Id
QueueTime = "${future}T23:00:00"
QueueTimeExpiry = "2021-03-11T05:00:00"
} | ConvertTo-Json
}
It will work fine.
So I am not really sure what I am missing here. Please let me know if there is something I am doing wrong. I believe it has to do with the Array, because if I comment that line out the issue will go away, but I need a way to transform the $future variable to adding one day to it.
Thanks
This should explain your error:
PS /home/> $future=[array]'2021-03-11'
PS /home/> (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
Get-Date: Cannot convert 'System.Object[]' to the type 'System.DateTime' required by parameter 'Date'. Specified method is not supported.
PS /home/> $future=[datetime]'2021-03-11'
PS /home/> (get-date $future).Adddays(1).ToString("yyyy-MM-dd")
2021-03-12
$password = ConvertTo-SecureString "xxx" -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential "xxx#xxx.onmicrosoft.com",$password
$url = "https://outlook.office365.com/api/v1.0/me/messages"
$today = (Get-Date).AddDays(1).ToString("yyyy-MM-dd")
## Set date and query
$messageQuery = "" + $url + "?`$select=Id&`$filter=HasAttachments eq true and DateTimeReceived lt " + $today
$messages = Invoke-RestMethod $messageQuery -Credential $cred
## Loop through each results
foreach ($message in $messages.value)
{
# get attachments and save to file system
$query = $url + "/" + $message.Id + "/attachments"
$attachments = Invoke-RestMethod $query -Credential $cred
# in case of multiple attachments in email
foreach ($attachment in $attachments.value)
{
$attachment.Name
$path = "C:\path\" + $attachment.Name
$Content = [System.Convert]::FromBase64String($attachment.ContentBytes)
Set-Content -Path $path -Value $Content -Encoding Byte
}
}
When this is run only 10 attachments are downloaded then it stops.
I would like to figure out the reason why it stops at 10 so that i could go ahead and download an entire inbox.
I am having the same issue. I think it only handles X amount at a time based on attachment size. I delete mine out of the mailbox when I have successfully copied the attachment with these lines, since I do not need the email after I have harvested the attachments.
This (with the rest of my code) runs in Task Scheduler every 5 minutes, so if it doesn't catch all the mail in the first run, they will be processed, I guess 10 at a time in subsequent runs.
If ($DeleteEmail -eq "Y") {
$query2 = $url + "/" + $message.Id
Invoke-RestMethod $query2 -Credential $Credentials -Method Delete
}