so I'm really new to working with the Google Analytics API. I have managed to make the request work:
{
"dateRange": {
"startDate": "2021-01-08",
"endDate": "2021-05-05"
},
"activityTypes": [
"GOAL"
],
"user": {
"type": "CLIENT_ID",
"userId": "2147448080.1620199617"
},
"viewId": "1556XXX89"
}
such that I can get back a json format file like:
{
"sessions": [
{
"sessionId": "1620199614",
"deviceCategory": "mobile",
"platform": "Android",
"dataSource": "web",
"activities": [
{
"activityTime": "2021-05-05T07:53:08.366983Z",
"source": "(direct)",
"medium": "(none)",
"channelGrouping": "Direct",
"campaign": "(not set)",
"keyword": "(not set)",
"hostname": "somewebsite.com",
"landingPagePath": "/client/loginorcreate/login",
"activityType": "GOAL",
"customDimension": [
{
"index": 1
},
{
"index": 2
},
{
"index": 3,
"value": "59147"
}
],
"goals": {
"goals": [
{
"goalIndex": 1,
"goalCompletions": "1",
"goalCompletionLocation": "/order/registerorder/postregister.html",
"goalPreviousStep1": "page-z",
"goalPreviousStep2": "page-y",
"goalPreviousStep3": "page-x",
"goalName": "order"
}
]
}
}
],
"sessionDate": "2021-05-05"
}
],
"totalRows": 1,
"sampleRate": 1
}
now, ideally, I would get a different format response, and more importantly on where I don't have to specify each individual client ID. is there such a request format I can build, which would return something like:
clientID1 | activityTime | sessionId | activities
clientID2 | activityTime | sessionId | activities
clientID3 | activityTime | sessionId | activities
thanks!
The google analytics api returns data in the json format that you are seeing currently. How you format that data will be up to you this must however been done locally by you.
Related
I need to combine values from 2 JSONs:
If there is a match in alerts IDs, I need to create structure, that will take data from both jsons
Result for a match should look like:
$array = #()
$hashtable = #{}
$hashtable.AlertID (does not matter what JSON is it from)
$hashtable.Tags (from JSON 1)
$hashtable.IncidentName (from JSON2)
$hashtable.IncidentID (from JSON2)
$array += $hashtable
I would prefer if this would be done with c style powershell loop.
c style for loop = for ($x = 0; $x -array.count; $x++)
JSON 1:
[
{
"Status": "Active",
"IncidentId": "3",
"tags": "SINC0008009",
"AlertId": [
"da637563185629568182_-638872186",
"da637563185631732095_1120592736",
"da637563185706412029_-614525914",
"da637563185760439486_-276692370",
"da637563185856325888_-1949235651",
"da637563186785996176_2128073884",
"da637563186789897000_1239551047",
"da637563186806513555_1512241399",
"da637563193194338043_-244132089"
],
"severity": "Medium"
},
{
"Status": "Active",
"IncidentId": "4",
"tags": "SINC0008008",
"AlertId": [
"da637643650725801726_1735022501",
"da637643650741237104_1473290917",
"da637643650748739479_-40211355",
"da637643652767933265_-1887823168",
"da637643670830160376_-443360743"
],
"severity": "Medium"
},
{
"Status": "Active",
"IncidentId": "2",
"tags": null,
"AlertId": [
"caD76232A5-F386-3C5D-94CD-7C82A7F778DC"
],
"severity": "Medium"
},
{
"Status": "Active",
"IncidentId": "1",
"tags": null,
"AlertId": [
"ca6534FF45-D62A-3FB7-BD6B-FF5029C553DB"
],
"severity": "Medium"
}
]
JSON 2:
{
"value": [
{
"incidentId": 3,
"incidentName": "Multi-stage incident involving Initial access & Discovery on one endpoint",
"status": "Active",
"severity": "Medium",
"tags": ["SINC0000001"],
"comments": [],
"alerts": [
{
"alertId": "da637563185629568182_-638872186",
"incidentId": 3,
"description": "A suspicious PowerShell activity was observed on the machine. ",
"status": "New",
"severity": "Medium",
"devices": [
{
"deviceDnsName": "xxxxx"
}
],
"entities": [
{
"entityType": "User",
"accountName": "xxxxxx",
"userPrincipalName": "xxx#xx.xx"
},
{
"entityType": "Process"
},
{
"entityType": "Process",
"verdict": "Suspicious"
},
{
"entityType": "File"
}
]
},
{
"alertId": "da637563185631732095_1120592736",
"incidentId": 3,
"devices": [
{
"osPlatform": "Windows10",
"version": "1909"
}
],
"entities": [
{
"entityType": "User",
"remediationStatus": "None"
}
]
}
]
},
{
"incidentId": 4,
"incidentName": "Multi-stage incident involving Initial access & Discovery on one endpoint",
"status": "Active",
"severity": "Medium",
"tags": ["SINC0000002"],
"comments": [],
"alerts": [
{
"alertId": "da637563185629568182_-638872186",
"incidentId": 3,
"description": "A suspicious PowerShell activity was observed on the machine. ",
"status": "New",
"severity": "Medium",
"devices": [
{
"deviceDnsName": "xxxxx"
}
],
"entities": [
{
"entityType": "User",
"accountName": "xxxxxx",
"userPrincipalName": "xxx#xx.xx"
},
{
"entityType": "Process"
},
{
"entityType": "Process",
"verdict": "Suspicious"
},
{
"entityType": "File"
}
]
},
{
"alertId": "da637563185631732095_1120592736",
"incidentId": 3,
"devices": [
{
"osPlatform": "Windows10",
"version": "1909"
}
],
"entities": [
{
"entityType": "User",
"remediationStatus": "None"
}
]
}
]
}
]
}
Till now, I was looking into using nested foreach loop to address it but it does not behave like I want. I am looking for for loop as I could use the indexes.
Instead of creating an array of Hashtables, I think it's better to create an array of PsCustomObjects, because outputting the result to console/file/json would be a lot easier then.
$json1 = Get-Content -Path 'X:\json1.json' -Raw | ConvertFrom-Json
$json2 = Get-Content -Path 'X:\json2.json' -Raw | ConvertFrom-Json
$result = foreach ($incident in $json1) {
foreach ($alertId in $incident.AlertId) {
$json2.value | Where-Object { $_.alerts.alertId -eq $alertId } | ForEach-Object {
# output an object with the wanted properties
[PsCustomObject]#{
AlertID = $alertId # from json1
Tags = $incident.Tags # from json1
IncidentName = $_.incidentName # from json2
IncidentID = $_.incidentId # from json2
}
}
}
}
# output on screen
$result | Format-Table -AutoSize # or use Out-GridView
# output to new JSON
$result | ConvertTo-Json
# output to CSV file
$result | Export-Csv -Path 'X:\incidents.csv' -NoTypeInformation
Using your examples, the output to console window is:
AlertID Tags IncidentName IncidentID
------- ---- ------------ ----------
da637563185629568182_-638872186 SINC0008009 Multi-stage incident involving Initial access & Discovery on one endpoint 3
da637563185629568182_-638872186 SINC0008009 Multi-stage incident involving Initial access & Discovery on one endpoint 4
da637563185631732095_1120592736 SINC0008009 Multi-stage incident involving Initial access & Discovery on one endpoint 3
da637563185631732095_1120592736 SINC0008009 Multi-stage incident involving Initial access & Discovery on one endpoint 4
I was trying to work out the best way to filter out some pagerules data from Cloudflare, and while I've got a solution I'm looking at how ugly it is and thinking "there has to be a simpler way to do this."
I'm specifically asking about a better way to achieve the following goal using jq. I understand there are programming libraries I could use to accomplish the same, but the point of this question is to get a better understanding of how jq is intended to work.
Say I've got a long list of CloudFlare pagerules records, here are a few entries as a minimal example:
{
"example.org": [
{
"id": "341",
"targets": [
{
"target": "url",
"constraint": {
"operator": "matches",
"value": "http://ng.example.org/*"
}
}
],
"actions": [
{
"id": "always_use_https"
}
],
"priority": 12,
"status": "active",
"created_on": "2017-11-29T18:07:36.000000Z",
"modified_on": "2020-09-02T16:09:03.000000Z"
},
{
"id": "406",
"targets": [
{
"target": "url",
"constraint": {
"operator": "matches",
"value": "http://nz.example.org/*"
}
}
],
"actions": [
{
"id": "always_use_https"
}
],
"priority": 9,
"status": "active",
"created_on": "2017-11-29T18:07:55.000000Z",
"modified_on": "2020-09-02T16:09:03.000000Z"
},
{
"id": "427",
"targets": [
{
"target": "url",
"constraint": {
"operator": "matches",
"value": "nz.example.org/*"
}
}
],
"actions": [
{
"id": "ssl",
"value": "flexible"
}
],
"priority": 8,
"status": "active",
"created_on": "2017-11-29T18:08:00.000000Z",
"modified_on": "2020-09-02T16:09:03.000000Z"
}
]
}
What I want to do is extract the urls nested in the constraint.value fields for the always_use_https actions. The goal is to extract the values and return them as a json array. What I came up with is this:
jq '[
[
[
[
.[] | .[] | select(.actions[].id | contains("always_use_https"))
] | .[].targets[] | select(.target | contains("url"))
] | .[] | .constraint | select(.operator | contains("matches"))
] | .[].value
]'
Against our example this produces:
[
"http://ng.example.org/*",
"http://nz.example.org/*"
]
Is there a more succinct way to achieve this in jq?
This produces the expected output in accordance with the criteria as I understand them:
jq '.["example.org"]
| map(select( any(.actions[]; .id == "always_use_https"))
| .targets[]
| select(.target == "url")
| .constraint.value )
' cloudfare.json
I'm using loopback 3 to build a backend with mongoDB.
So i have 2 models: Object and Attachment. Object have a relation Embeds2Many to Attachment.
Objects look like that in mongoDB
[
{
"fieldA": "valueA1",
"attachments": [
{
"id": 1,
"url": "abc.com/image1"
},
{
"id": 2,
"url": "abc.com/image2"
}
]
},
{
"fieldA": "valueA2",
"attachments": [
{
"id": 4,
"url": "abc.com/image4"
},
{
"id": 5,
"url": "abc.com/image5"
}
]
}
]
The question is: how can i get Objects with attachments.id=4 over the RestAPI?
I have try with the where and include filter. But it didn't work. It look like, that this function is not implemented in loopback3, right?
I have found the solution. It only works on Mongodb, Cloudant and Memory database.
{
"filter": {
"where": {
"attachments.id": 4
}
}
}
I'm forced to use the aggregation framework and the project operation of Spring Data MongoDb.
What I'd like to do is creating an array of object as a result of a project operation.
Considering this intermediate aggregation result:
{
"processes": [
{
"id": "101a",
"assignees": [
{
"id": "201a",
"username": "carl93"
},
{
"id": "202a",
"username": "susan"
}
]
},
{
"id": "101b",
"assignees": [
{
"id": "201a",
"username": "carl93"
},
{
"id": "202a",
"username": "susan"
}
]
}
]
}
I'm trying to get for each process, all the assignee usernames and ids. Hence, what I want to obtain is something like this:
[
{
"results": [
{
"id": "201a",
"value": "carl93",
"parentObjectId": "101a"
},
{
"id": "202a",
"value": "susan",
"parentObjectId": "101a"
},
{
"id": "201a",
"value": "carl93",
"parentObjectId": "101b"
},
{
"id": "202a",
"value": "susan",
"parentObjectId": "101b"
}
]
}
]
To reach this goal I'm using 2 nested VariableOperators.mapItemsOf obtaining:
org.springframework.data.mapping.MappingException: Cannot convert [Document{{id= 201a, value= carl93, parentObjectId= 101a}}, Document{{id= 202a, value = susan, parentObjectId= 101a}}]
of type class java.util.ArrayList into an instance of class java.lang.Object!
Implement a custom Converter<class java.util.ArrayList, class java.lang.Object> and register it with the CustomConversions.
Here's the code that I'm currently using:
new ProjectionOperation().and(
VariableOperators.mapItemsOf("processes")
.as("pr")
.andApply(
VariableOperators.mapItemsOf("$pr.ownership.assignees")
.as("ass")
.andApply(aggregationOperationContext -> {
Document document = new Document();
document.append("id", "$$ass.id");
document.append("value", "$$ass.username");
document.append("parentObjectId", "$$pr.id");
return document;
})
)
).as("results");
The code produces this:
[
[
{
"id": "201a",
"value": "carl93",
"parentObjectId": "101a"
},
{
"id": "202a",
"value": "susan",
"parentObjectId": "101a"
}
],
[
{
"id": "201a",
"value": "carl93",
"parentObjectId": "101b"
},
{
"id": "202a",
"value": "susan",
"parentObjectId": "101b"
}
]
]
As you can see there are 2 nested arrays, [[],[]]. This is the reason why the exception is thrown.
Nevertheless what I want to obtain is just one array, adding all the objects in it (possibly without duplicates or null values). I've tried the addToSet operator and other aggregtion operators, without any success.
Use $reduce with $concatArrays to join the arrays.
new ProjectionOperation().and(
ArrayOperators.arrayOf("processes")
.reduce(ArrayOperators.ConcatArrays.arrayOf("$$value").concat(
VariableOperators.mapItemsOf("$$this.ownership.assignees")
.as("ass")
.andApply(aggregationOperationContext -> {
Document document = new Document();
document.append("id", "$$ass.id");
document.append("value", "$$ass.username");
document.append("parentObjectId", "$$this.id");
return document;
})
)).startingWith(Arrays.asList())
).as("results");
I'm trying to setup Teamcity building and verifying patchsets from Gerrit. The last step should set Verified to -1 if build failed. I'm playing around with Gerrit REST API and I think I found a proper command:
https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#set-review
The documentation says:
As response a ReviewInfo entity is returned that describes the applied
labels.
My request looks like this:
POST <gerrit-url>/a/changes/I696f00f4968fcb35fa614ce6325499aa15367150/revisions/current/review
{
"message": "Build failed",
"labels": {
"Verified": -1
}
}
As a response I get full revision info:
{
"id": "dev_test~master~<change-id>",
"project": "dev_test",
"branch": "master",
"hashtags": [],
"change_id": "<change-id>",
"subject": "a test",
"status": "NEW",
"created": "2017-04-03 07:53:19.000000000",
"updated": "2017-04-04 08:47:34.000000000",
"submit_type": "MERGE_IF_NECESSARY",
"mergeable": true,
"insertions": 133,
"deletions": 7,
"unresolved_comment_count": 0,
"_number": 381,
"owner": {
"_account_id": 4,
"name": "<my-name>",
"email": "<my-email>",
"username": "<my-username>",
},
"labels": {
"Code-Review": {
"all": [
{
"value": 1,
"date": "2017-04-04 08:47:34.000000000",
"permitted_voting_range": {
"min": -2,
"max": 2
},
"_account_id": 4,
"name": "<my-name>",
"email": "<my-email>",
"username": "<my-username>"
}
],
"values": {
"-2": "This shall not be merged",
"-1": "I would prefer this is not merged as is",
" 0": "No score",
"+1": "Looks good to me, but someone else must approve",
"+2": "Looks good to me, approved"
},
"default_value": 0
},
"Verified": {
"all": [
{
"value": 0,
"permitted_voting_range": {
"min": -1,
"max": 1
},
"_account_id": 4,
"name": "<my-name>",
"email": "<my-email>",
"username": "<my-username>"
}
],
"values": {
"-1": "Fails",
" 0": "No score",
"+1": "Verified"
},
"default_value": 0
}
},
"permitted_labels": {},
"removable_reviewers": [],
"reviewers": {
"REVIEWER": [
{
"_account_id": 4,
"name": "<my-name>",
"email": "<my-email>",
"username": "<my-username>"
}
]
},
"current_revision": "913330441711b067899a664a60c78be518e547b4",
"revisions": {
"913330441711b067899a664a60c78be518e547b4": {
"kind": "REWORK",
"_number": 6,
"created": "2017-04-03 14:08:14.000000000",
"uploader": {
"_account_id": 4,
"name": "<my-name>",
"email": "<my-email>",
"username": "<my-username>"
},
"ref": "refs/changes/81/381/6",
"fetch": {
"ssh": {
"url": "ssh://<url>",
"ref": "refs/changes/81/381/6"
},
"http": {
"url": "http://<url>",
"ref": "refs/changes/81/381/6"
}
}
}
}
}
It's not affected by request. Same response is returned when I send request using GET method or using POST method with invalid JSON in body(!)
Gerrit version is: 2.13.6-3008-gcdc381e
Do I something wrong?
PS. Here is similar question, but it isn't helpful: Gerrit set-review api doesn't work
EDIT:
It seems that I getting response from GET request not POST
I figured it out. It's not gerrit problem. I used http request and our server redirected to https with 301 which the Postman fallowed and returned response for GET request.