How to append web request results in a while loop - powershell

I need to call an API and loop through the various pages of results that are returned and append them all to one object.
I've tried the code below. Generally += works when appending to a powershell object, but no luck this time.
Note: URI and Get are both functions that are defined elsewhere. They work as expected elsewhere in the code.
$min=1
$max=2
while ($min -le $max){
$url= URI "tasks?page=$min"
$x=Get $url
if($min=1){
$response=$x
}
else{
$response+=$x
}
$min=$min+1
}
sample response (converted to json):
"value": [
{
"task_id": 17709655,
"project_id": 1928619,
"start_date": "2019-04-11",
"end_date": "2019-11-29",
"start_time": null,
"hours": 1.5,
"people_id": 17083963,
"status": 2,
"priority": 0,
"name": "",
"notes": "",
"repeat_state": 0,
"repeat_end_date": null,
"created_by": 331791,
"modified_by": 0,
"created": "2019-04-12 00:39:30.162",
"modified": "2019-04-12 00:39:30.162",
"ext_calendar_id": null,
"ext_calendar_event_id": null,
"ext_calendar_recur_id": null
},
{
"task_id": 17697564,
"project_id": 1928613,
"start_date": "2019-10-08",
"end_date": "2019-10-08",
"start_time": null,
"hours": 8,
"people_id": 17083966,
"status": 2,
"priority": 0,
"name": "",
"notes": "",
"repeat_state": 0,
"repeat_end_date": null,
"created_by": 327507,
"modified_by": 0,
"created": "2019-04-11 16:10:22.969",
"modified": "2019-04-11 16:10:22.969",
"ext_calendar_id": null,
"ext_calendar_event_id": null,
"ext_calendar_recur_id": null
}
],
"Count": 2
}```

Assuming you want the output to be an array, I'd write your code like this:
$min=1
$max=2
$response = foreach ($Page in $min..$max) {
$url = URI "tasks?page=$Page"
Get $url
}
This is the generally preferred method, because both Strings and Arrays have fixed lengths in .Net and therefore PowerShell.
Here, $response[0] should be the first response, $response[1] the second, etc.
If the above doesn't work for you, then my first guess would be that the output of Get isn't a string.
If you're expecting $response to be a single valid JSON string containing all the responses, then my response is "JSON doesn't work that way." You'll have to parse each JSON response to objects (hint: ConvertFrom-Json) combine them, and then possibly convert them back to JSON (ConvertTo-Json). Note that .Net's native dialect of JSON doesn't match the rest of the Internet's dialect of JSON, particularly with dates (though it looks like your dates here are strings). You may want to use JSON.Net, which I believe does match the common Internet dialect.
You may be able to combine $response like this:
$CombinedResponse = '[' + ($response -join ',') + ']'
But I don't know how well that's going to work if you then try to parse that as JSON.

Related

issues PowerShell POST request with body

I have a bit issue with powershell + invoke-webrequest
This is my body
$pram = #{
"name": "MDE.Windows",
"id": "$resourceId/extensions/MDE.Windows",
"type": "Microsoft.Compute/virtualMachines/extensions",
"location": "westeurope",
"properties": {
"autoUpgradeMinorVersion": true,
"publisher": "Microsoft.Azure.AzureDefenderForServers",
"type": "MDE.Windows",
"typeHandlerVersion": "1.0",
"settings": {
"azureResourceId": "$resourceId",
"defenderForServersWorkspaceId": "$subscriptionId",
"vNextEnabled": "true",
"forceReOnboarding": true,
"provisionedBy": "Manual"
},
"protectedSettings": {
"defenderForEndpointOnboardingScript": "$defenderForEndpointOnboardingScript"
}
}
}
I don't get watch wrong with my body because by looking examples from google this should be right but it still ouputs red
I have tried also with #"{ }"#, #'{ }'#, { }, "{ }" but no matter what I do it is more or less red.
I think your mistaking powershell hash tables for json. Normally you would create a hashtable using powershell syntax, then convert that object into Json. eg
$pram = #{
name= "MDE.Windows";
id= "$resourceId/extensions/MDE.Windows";
} | ConvertTo-Json
You can now pass the json encoded value of $parm to Invoke-WebRequest.
The other option is to create a String and write the JSON yourself:
$paramString = '{"id": "/extensions/MDE.Windows", "name": "MDE.Windows"}'
(but the first solution is probably the solution your looking for).

data factory - check if result of pipeline task contains a specific string or value?

I have a web task in a pipeline.
The result of the api call can be one of 3:
An item exists in a database.
an item does not exist in a database.
the body sent to the call was invalid.
If the record exists, I want to take further action.
As the resulting JSON from the API call is entirely different when an entity exists vs does not exists, how can I check for if a specific value is returned, if there is a chance it wont be returned at all?
Below is the output from a successful call. I'd like to check for the "Tag" attribute, but also cover a case where this does not exist.
How is this done using a dynamic content expression, seeing as there is no exists function?
{
"Data": [
{
"Data": {
"RowNumber": 0,
"Tag": "GLT-GM-45",
"GMStatus": 0,
"CustomValues": [
{
"CSLabel": "Asset Status",
"CSType": 0,
"CSValue": "Assigned"
},
{
"CSLabel": "Usage Status",
"CSType": 0,
"CSValue": "Permanent"
}
],
"AttachmentsToAdd": [],
"AttachmentsToDelete": []
},
"Messages": [
{
"ResultCode": 0,
"Message": "Success",
"HttpStatusCode": 200,
"FieldName": "GLT-GM-45"
}
],
"HasError": false,
"HasHttpError": false,
"HasMessage": true,
"HasSuccessWithMoreDataRemaining": false
}
],
"Messages": [
{
"ResultCode": 0,
"Message": "Success.",
"HttpStatusCode": 200,
"FieldName": ""
}
],
"TotalRecordsLongCount": 1,
"HasSuccessWithMoreDataRemaining": false,
"HasError": false,
"HasMessage": true,
"HasHttpError": false,
"ADFWebActivityResponseHeaders": {
"Connection": "keep-alive",
"Pragma": "no-cache",
"WaspResult": "WaspResult",
"X-UA-Compatible": "IE=edge;IE=edge",
"Cache-Control": "no-store, no-cache",
"Date": "Tue, 02 Aug 2022 11:58:31 GMT",
"Server": "Microsoft-IIS/10.0;Microsoft-IIS/8.5",
"X-AspNet-Version": "4.0.30319",
"X-Powered-By": "ASP.NET;ASP.NET",
"Content-Length": "2364",
"Content-Type": "application/json",
"Expires": "-1"
},
"effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime (North Europe)",
"executionDuration": 0,
"durationInQueue": {
"integrationRuntimeQueue": 1
},
"billingReference": {
"activityType": "ExternalActivity",
"billableDuration": [
{
"meterType": "AzureIR",
"duration": 0.016666666666666666,
"unit": "Hours"
}
]
}
}
Wouldn't you be able to use an IF Condition that uses a contains function? Obviously I do not have a way to recreate your output, but I did the following to simulate something.
Added an If Condition to a pipeline
Added dynamic content that uses a contains function. The first part was a string containing part of your output from above (copy/paste). The part behind the comma (i.e. the search string) being 'Tag'
Added False activity causing the component to fail
Added a True activity to set a variable to be equal to "Found"!.
The full If Condition:
#contains('{
"Data": [
{
"Data": {
"RowNumber": 0,
"Tag": "GLT-GM-45",
"GMStatus": 0
}
}', 'Tag')
Seeing the 'Tag' should be found in the string, it produces the result of setting the variable:
Changing the search word to NotATag which does not exist in the string produces this instead:
My guess would be that you can substitute the hard-coded string with something like the following:
#contains(activity('Web1').output, 'Tag')
Obviously, change 'Web1' to the name of your web component.
Might not be an ideal way but you can do something like below to check the existence of Tag attribute in output (the reason i am trying to check the string size is the pipeline exp expected the return type to be boolean, hence using condition to achieve it), you can map the other actions based on If activity status. If the property doesn't exist if condition fails with error.
Condition
Mapping

POSTing a nested object - returning the nested object inside the parent

Suppose that we have a Post resource (this is just a dummy example):
GET /api/posts/1
{
"id": 1,
"header": null,
"content": null
}
And now we'd like to create a Header inside the Post#1.
POST /api/posts/1/header
{
"color": "blue",
"title": "Some title"
}
Now, is it okay for the POST request above to return the following response?
{
"id": 1,
"header": {
"id": 1,
"color": "blue",
"title": "Some title"
},
"content": null
}
So, basically, a Header was created inside the Post and returned as part of the Post.
Also, if we need to GET the header for Post#1:
GET api/posts/1/header
{
"id": 1,
"color": "blue",
"title": "Some title"
}
So here only the Header is returned.
Edit: Formatted JSON following #Mike Slinn response.
Nested JSON objects are not a problem.
The problems with the JSON that you show are:
Single quotes are used instead of double quotes. Perhaps you are writing in Python and serializing a dict into JSON? If so, no problem. Otherwise, you need to replace the single quotes with double quotes.
Similarly, the keys need to be double-quoted.
{
"id": 1,
"color": "blue",
"title": "Some title"
}

Retriving child nodes from JSON response in PowerShell

{
"completionTime": 1477067415024,
"context": {
"environmentId": 78
},
"id": 51,
"jobId": 473,
"jobName": "Ravindra",
"reportIds": [
959
],
"startTime": 1477067357196,
"status": "PASSED",
"username": "svc.soaess"
}
from this json structure I have to get the value 959.
$response.ChildNodes.reportIds displays blank. Tried with array etc. no luck.
Presumably $response is a string, so you need to convert it to an object first, before you can access the object's properties.
($response | ConvertFrom-Json).reportIds

Does the OData protocol provide a way to transform an array of objects to an array of raw values?

Is there a way specify in an OData query that instead of certain name/value pairs being returned, a raw array should be returned instead? For example, if I have an OData query that results in the following:
{
"#odata.context": "http://blah.org/MyService/$metadata#People",
"value": [
{
"Name": "Joe Smith",
"Age": 55,
"Employers": [
{
"Name": "Acme",
"StartDate": "1/1/1990"
},
{
"Name": "Enron",
"StartDate": "1/1/1995"
},
{
"Name": "Amazon",
"StartDate": "1/1/1999"
}
]
},
{
"Name": "Jane Doe",
"Age": 30,
"Employers": [
{
"Name": "Joe's Crab Shack",
"StartDate": "1/1/2007"
},
{
"Name": "TGI Fridays",
"StartDate": "1/1/2010"
}
]
}
]
}
Is there anything I can add to the query to instead get back:
{
"#odata.context": "http://blah.org/MyService/$metadata#People",
"value": [
{
"Name": "Joe Smith",
"Age": 55,
"Employers": [
[ "Acme", "1/1/1990" ],
[ "Enron", "1/1/1995" ],
[ "Amazon", "1/1/1999" ]
]
},
{
"Name": "Jane Doe",
"Age": 30,
"Employers": [
[ "Joe's Crab Shack", "1/1/2007" ],
[ "TGI Fridays", "1/1/2010" ]
]
}
]
}
While I could obviously do the transformation client side, in my use case the field names are very large compared to the data, and I would rather not transmit all those names over the wire nor spend the CPU cycles on the client doing the transformation. Before I come up with my own custom parameters to indicate that the format should be as I desire, I wanted to check if there wasn't already a standardized way to do so.
OData provides several options to control the amount of data and metadata to be included in the response.
In OData v4, you can add odata.metadata=minimal to the Accept header parameters (check the documentation here). This is the default behaviour but even with this, it will still include the field names in the response and for a good reason.
I can see why you want to send only the values without the fields name but keep in mind that this will change the semantic meaning of the response structure. It will make it less intuitive to deal with as a json record on the client side.
So to answer your question, The answer is 'NO',
Other options to minimize the response size:
You can use the $value OData option to gets the raw value of a single property.
Check this example:
services.odata.org/OData/OData.svc/Categories(1)/Products(1)/Supplier/Address/City/$value
You can also use the $select option to cherry pick only the fields you need by selecting a subset of properties to include in the response