I am sending a GET request that returns an object. Using postman, I get:
{
"changedInstance": {
"change": "Created",
"instanceAfterChange": {
"instanceId": "123",
"schemaName": "RBAC",
"className": "PermissionCategory",
"properties": {
"Name": "IModel Schema Management Service",
"TypeId": 2,
"TypeName": "Service Access and Permissions"
}
}
}}
But when I send a request through Powershell, I get the response:
#{change=Created; instanceAfterChange=}
The request I have made:
$Response = Invoke-RestMethod -Uri $base_url/v2.4/Repositories/BentleyCONNECT--Main/RBAC/PermissionCategory -Method Post -Headers #{'Authorization' = 'Bearer ' + $global:token} -Body $postParams -ContentType "application/json"
I have checked all the header values and the body, no values are missing.
The property 'instanceAfterChange' is empty, however when I monitor this same request in fiddler, it shows the correct response. My question is, why is Powershell displaying it as empty even though when it is not.
ConvertFrom-Json constructs a [pscustomobject] instance from your JSON input, and what you're seeing (#{change=Created; instanceAfterChange=}) is the default display formatting for the instance's .changedInstance property value.
Due to a bug as of PowerShell 7.0, the default display formatting makes it appear as if property instanceAfterChange is empty, but it actually isn't.
Specifically, the bug is that calling .ToString() on [pscustomobject] instances unexpectedly yields the empty string - see this GitHub issue.
The hashtable-like display representation of [pscustomobject]s as property values - as shown in your question and explained in this answer - involves .ToString() calls on nested [pscustomobject] instances, which is why the bug surfaces.
Therefore, you shouldn't have a problem accessing the nested [pscustomobject]s' properties, e.g., $convertedFromJson.changedInstance.instanceAfterChange.schemaName should yield RBAC.
An easy way to verify that the data is there is to convert back to JSON (ConvertTo-Json) or to visualize the structure via Format-Custom.
A simplified example:
$convertedFromJson = '{
"changedInstance": {
"instanceAfterChange": {
"instanceId": "123",
"schemaName": "RBAC"
}
}
}
' | ConvertFrom-Json
# Output as-is: the bug makes .changedInstance appear empty.
$convertedFromJson
'---'
# Access a nested property - this works.
$convertedFromJson.changedInstance.instanceAfterChange.schemaName
'---'
# Visualize the structure of the nested object.
$convertedFromJson | Format-Custom
The above yields:
changedInstance
---------------
#{instanceAfterChange=} # DISPLAY BUG: .instanceAfterChange appears empty.
---
RBAC # OK: accessing a property of .instanceAfterChange works.
---
# Visualization of the nested object's structure via Format-Custom
class PSCustomObject
{
changedInstance =
class PSCustomObject
{
instanceAfterChange =
class PSCustomObject
{
instanceId = 123
schemaName = RBAC
}
}
}
Related
So, I have a code like the following.
$x = Invoke-PowerBIRestMethod -Url 'datasets/<dataset_id>/refreshes' -Method Get
When I run it, the returned JSON data is like this,
{
"#odata.context":"<some_url>/myorg/$metadata#refreshes","value":[
{
....
},
{
...
}
]
}
But if I try to get the field value, by typing $x.value, I get nothing. What mistake am I doing. I am new to PowerShell.
The output was in string form. I had to first convert it to JSON form by using the following command.
$j = $x | ConvertFrom-Json
Then I can see the value by typing $j.value
I can successfully make POST requests in Postman using this body in my call, which uses nested key pair values:
{
"meta": null,
"data": {
"type": "spaces_schema"
"attributes": {
"spaces": ["space1", "space2"]
}
}
}
I am transposing the call into PowerShell, and creating the body using a nested hash table. The call returns an error regarding the body when done in PowerShell:
$Body = #{
"meta" = "null"
"data" = #{
"type" = "spaces_schema"
"attributes" = #{
"spaces" = ["space1", "space2"]
}
}
}
The problem I am running into is the "spaces" attribute. I am not sure how to pass the brackets as literal characters in the hash table without wrapping them in quotes, since doing so puts out an error as well and interferes with the quotes used inside of the brackets.
Here is the error:
Invoke-WebRequest : {"errors":[{"additional_data":null,"detail":"Unhandled exception from route request: Expecting value: line 1 column 1 (char 0)"}]}
At C:\Users\{user}\Desktop\{Folder}\{Scriptname}.ps1:26 char:1
+ Invoke-WebRequest -URI 'https://{URL} ...
You are trying too hard to make the object look like JSON syntax.
As said, to define an element's value as array, you need to enclose it in #(). In Json syntax that would display inside square brackets.
Writing "meta" = "null" is wrong too, because you then define it as string with a literal value of "null".
What you should do in the object is meta = $null which wil translate to JSON as "meta": null
In your case, the default -Depth setting of the ConvertTo-Json cmdlet is not deep enough, so you need to set that to a value of at least 3 (in the example you gave).
Try
$Body = #{
meta = $null
data = #{
type = "spaces_schema"
attributes = #{
spaces = #("space1", "space2")
}
}
}
$Body | ConvertTo-Json -Depth 3
Since you will be uploading this data using Invoke-WebRequest or Invoke-RestMethod, I would advice to add switch -Compress to the ConvertTo-Json cmdlet as well. This will return a less human readable json, but the payload will be smaller because all extra whitespace will be removed:
{"meta":null,"data":{"type":"spaces_schema","attributes":{"spaces":["space1","space2"]}}}
I am using Knack to solve a business process issue, however to ensure it's success I need to run a daily script to sync the data between Knack and our HR system.
Knack uses a REST API and I want to apply a filter to the GET call so that I don't have to populate a local table in order to compare the data.
Knack requires a JSON array like the example below to be encoded in the URL in order to filter the results returned. The example shows how to do this in Javascript but I cannot work out how to do this in Powershell would anyone be able to assist?
// Request route
var api_url = 'https://api.knack.com/v1/objects/object_1/records';
// Prepare filters
var filters = {
'match': 'or',
'rules': [
{
'field':'field_1',
'operator':'is',
'value':'Dodgit'
},
{
'field':'field_1',
'operator':'is blank'
}
]
};
// Add filters to route
api_url += '?filters=' + encodeURIComponent(JSON.stringify(filters));
You can do the following:
$api_url = 'https://api.knack.com/v1/objects/object_1/records'
$filters = #'
{
'match': 'or',
'rules': [
{
'field':'field_1',
'operator':'is',
'value':'Dodgit'
},
{
'field':'field_1',
'operator':'is blank'
}
]
}
'#
$CompressedFilters = $filters | ConvertFrom-Json | ConvertTo-Json -Depth 10 -Compress
$encodedUri = "{0}?filters={1}" -f $api_url,[System.Web.HttpUtility]::UrlEncode($CompressedFilters)
Explanation:
$filters is defined as a here-string. $filters is piped into ConvertFrom-Json and then to ConvertTo-Json in order to compress the JSON data easily. However, that does swap single quotes for double quotes.
UrlEncode() from the System.Web.HttpUtility .NET class encodes special characters in URLs.
-f is the string format operator. This helps to build your URI string. The final URI is stored in $encodedUri.
NOTE: The [System.Web.HttpUtility]::UrlEncode encodes special characters with lowercase hex characters. If your system requires uppercase hex characters, then use [System.Net.WebUtility]::UrlEncode instead.
Over the past few days I've been looking for documentation on variable expansion in JSON objects. I've tried a lot of recommendations and fixes, but none of them seem to work quite as expected. I'm left wondering if there's a better way than I have implemented. Anyway, on to my use case.
Use Case
I have an array of integers, which represent objects in a web application. These objects are able to have a relationship established between them by making a POST request to an API endpoint.
This array of integers is referred to as $combined.
PS C:\> $combined
i_id m_id
------------ ----------
75419 6403
75420 6403
75421 6403
75410 6403
To set up the parameters used for my POST request, I've created another array called $associationParams.
$associationParams = #{
ContentType = "application/json"
Body = '{"id": '+$($thing).m_id+'}'
Method = "POST"
URI = "https://application/api/in/$($thing.i_id)/endpoint"
Headers = $headers
}
The $headers variable above is the authorization token.
So, my initial inclination is to loop through the array of integers ($combined) and call Invoke-RestMethod as so:
Foreach ($thing in $combined) {
Invoke-RestMethod #iocAssociationParams
}
Problem
So, I don't really have a problem per se... There seems to be some inconsistency in my testing where the final ForEach doesn't seem to loop through all the elements in the $combined array... But I am wondering if there's a better way. I've tried string formatting the JSON object in the Body parameter in $associationParams, but it definitely didn't work. I've also tried various forms of quoting, but they didn't work either. Is trying to splat the parameters for Invoke-RestMethod the best way? Should I just list out all the parameters for Invoke-RestMethod in the ForEach loop?
Use the -f format operator to insert $thing.m_id inside the foreach into $AssociationParams.
You have to escape the literal curly braces by doubling them.
Foreach ($thing in $combined) {
$AssociationParams = #{
ContentType = "application/json"
Body = '{{"id": {0}}}' -f $thing.m_id
Method = "POST"
URI = "https://application/api/in/$($thing.i_id)/endpoint"
Headers = $headers
}
Invoke-RestMethod #AssociationParams
}
Here's my solution as implemented.
Foreach ($thing in $combined) {
# Now we create the body of the request as a Powershell object with the intent to convert it to JSON later
$AssociationBodyObj = [PSCustomObject]#{
id = $($thing.id)
}
# ...and then we convert it to JSON
$AssociationBodyObj = $AssociationBodyObj | ConvertTo-Json
$iocAssociationParams = #{
ContentType = "application/json"
Body = $AssociationBodyObj
Method = "POST"
URI = "https://application/api/in/$($thing.i_id)/endpoint"
Headers = $headers
}
Invoke-RestMethod #AssociationParams
}
I cannot to seem to find any examples on this topic and I would like to know how to do it. Can anyone show me an example or point me to a link on how to do pagination in powershell with Invoke web-request? The challenge I am facing is that I am making API calls to a server that only returns 100 rows at a time. In order to get any more rows, I would have to make a second call to the server. I have no clue how to do it.
If it helps, here is the link provided by the Canvas LMS and my code that I have so far.
Pagination:
Pagination
Requests that return multiple items will be paginated to 10 items by
default. You can set a custom per-page amount with the ?per_page
parameter. There is an unspecified limit to how big you can set
per_page to, so be sure to always check for the Link header.
To retrieve additional pages, the returned Link headers should be
used. These links should be treated as opaque. They will be absolute
urls that include all parameters necessary to retrieve the desired
current, next, previous, first, or last page. The one exception is
that if an access_token parameter is sent for authentication, it will
not be included in the returned links, and must be re-appended.
Pagination information is provided in the Link header:
Link:
<https://<canvas>/api/v1/courses/:id/discussion_topics.json?opaqueA>; rel="current",
<https://<canvas>/api/v1/courses/:id/discussion_topics.json?opaqueB>;> rel="next",
<https://<canvas>/api/v1/courses/:id/discussion_topics.json?opaqueC>;> rel="first",
<https://<canvas>/api/v1/courses/:id/discussion_topics.json?opaqueD>;> rel="last"
The possible rel values are:
current - link to the current page of results. next - link to the next
page of results. prev - link to the previous page of results. first -
link to the first page of results. last - link to the last page of
results. These will only be included if they are relevant. For
example, the first page of results will not contain a rel="prev" link.
rel="last" may also be excluded if the total count is too expensive to
compute on each request.
The beginning product
$curlly=""
$url_main="https://[instance].instructure.com/api/v1/accounts/1/courses?per_page=1"
$security_token="imhungry"
$header = #{"Authorization"="Bearer "+ $security_token; "rel"="last"}
$curlly=Invoke-WebRequest -Headers $header -Method Get -Uri $url_main
$curlly = ConvertFrom-Json $curlly.Content
foreach($course in $curlly)
{
$course.name
}
$curlly.Count
The final product
##This is an example on how to use pagination in powershell
$url_main="https://[instance].instructure.com/api/v1/accounts/1/courses?per_page=100"
$security_token="boyimhungry"
$header = #{"Authorization"="Bearer "+ $security_token}
$purlly=Invoke-WebRequest -Headers $header -Method Get -Uri $url_main
$curlly = ConvertFrom-Json $purlly.Content
$url_main = $purlly.Headers.Link.Split(",")[1].Replace("<","").Replace(">","") ## you can get away with just doing one replace("<","") but it looks neater this way
while( !$url_main.Contains("prev"))
{
$purlly=Invoke-WebRequest -Headers $header -Method Get -Uri $url_main
$curlly += ConvertFrom-Json $purlly.Content
$url_main = $purlly.Headers.Link.Split(",")[1].Replace("<","").Replace(">","")
cls
$curlly.Count
$url_main
}
foreach($course in $curlly)
{
$course.name
}
$curlly.Count
I know you've accepted an answer, but I thought I'd give my code example just in case anybody needs one. This example is getting a list of all of our Canvas users. Not a terrible process -- most of the work is done with just a 4-line do..while loop.
$token = "YOUR_ACCESS_TOKEN"
$headers = #{"Authorization"="Bearer "+$token}
$allCanvasUsers = New-Object System.Collections.ArrayList #()
$pageNumber = 1
Function RequestPageOfUsers($page) {
$script:resultsPage = Invoke-WebRequest -Method GET -Headers $headers -Uri "https://$domain/api/v1/accounts/self/users?per_page=100&search_term=P00&page=$page"
$usersPage = ConvertFrom-Json $resultsPage.Content
foreach ($user in $usersPage) {
$script:allCanvasUsers.Add($user)
}
}
do {
RequestPageOfUsers $pageNumber
$pageNumber++
} while ($resultsPage.Headers.Link.Contains('rel="next"'))
David Baker's answer is fantastic, and nearly worked for me, but was only returning a single page. Not sure why, but the CONTAINS('...') test was always returning FALSE.
In order to get this to work, I had to make a minor change to the WHILE condition. For me, the ResultsPage object was returned with a property called RelationLink, which is a dictionary containing key-value pairs. The last page in a paginated set of results has no 'Next' key. So, my DO-WHILE loop runs until the 'Next' value is found to be null.
$Token = '<YOUR-TOKEN>'
$headers = #{"Authorization"="Bearer "+$token}
$allCanvasUsers = New-Object System.Collections.ArrayList #()
$pageNumber = 1
Function RequestPageOfUsers($page) {
$script:resultsPage = Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://<DOMAIN>:443/api/v1/accounts/self/users?per_page=100&page=$page"
$usersPage = ConvertFrom-Json $resultsPage.Content
foreach ($user in $usersPage) {
$script:allCanvasUsers.Add($user)
}
}
do {
RequestPageOfUsers $pageNumber
$pageNumber++
} while ($resultsPage.RelationLink.Next -NE $NULL)
This looks like a real pain.
On each paginated request, you're going to get back a Link header that contains one or more of the links they describe.
For your purposes (a sequential read through of every result), you only really need to be concerned with the link rel=next link. You would keep calling that one until there is no rel=next anymore, which is how you'll know that you're on the last page.
You don't provide the rel part; it's not a header. It's something that lets you identify which link to use, and then use that link as is.
So the basic overview of what you need to do:
Make the first request.
Read through the returned Link header, and find the one that corresponds to rel=next (if it's not there, you're done).
Make the next request directly to the link you found.
Parse the Link header again, repeat.