Source JSON:
[
{
"other": false,
"granularity": [
{
"impressions": 15,
"date": "2021-11-02"
},
{
"impressions": 67,
"date": "2021-11-03"
}
],
"metadata": {
"adGroupId": 5792423,
"startTime": "2021-06-29T21:00:00.000"
}
},
{
"other": false,
"granularity": [
{
"impressions": 226,
"date": "2021-11-02"
},
{
"impressions": 2339,
"date": "2021-11-03"
}
],
"metadata": {
"adGroupId": 578101600,
"startTime": "2021-06-29T08:05:52.176"
}
}
]
I tried with this spec:
[
{
"operation": "shift",
"spec": {
"*": {
"granularity": {
"*": "[&2].&"
},
"metadata": {
"adGroupId": "[&2].adGroupId"
}
}
}
}
]
Result:
[
{
"0": {
"impressions": 15,
"date": "2021-11-02"
},
"1": {
"impressions": 67,
"date": "2021-11-03"
},
"adGroupId": 5792423
},
{
"0": {
"impressions": 226,
"date": "2021-11-02"
},
"1": {
"impressions": 2339,
"date": "2021-11-03"
},
"adGroupId": 578101600
}
]
I want to remove these indexes 0,1,...etc from top level. And adGroupId should be in each row.
So I expect this result:
[
{
"impressions": 15,
"date": "2021-11-02",
"adGroupId": 5792423
},
{
"impressions": 67,
"date": "2021-11-03",
"adGroupId": 5792423
},
{
"impressions": 226,
"date": "2021-11-02",
"adGroupId": 578101600
},
{
"impressions": 2339,
"date": "2021-11-03",
"adGroupId": 578101600
}
]
How to fix it? Thanks!
You can collect the key-value pairs under the granularity key while prefixing the subkeys by [&3].&1. in order to pick the common indexes by going three levels up, and then remove the keys of the indexes at the last step such as
[
{
"operation": "shift",
"spec": {
"*": {
"granularity": {
"*": {
"*": "[&3].&1.&",
"#(2,metadata.adGroupId)": "[&3].&1.adGroupId"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
Related
I have this input JSON:
{
"user": "123456",
"product": "television",
"category": "electronics",
"tag": "summer"
}
And this transformation:
[
{
"operation": "shift",
"spec": {
"product": {
"#(1,product)": "item",
"#(1,user)": {
"#2": "userBias"
}
},
"user": {
"#(1,user)": "user"
},
"category": {
"#category": "rules.[0].name",
"#(1,category)": "rules.[0].values[0]"
},
"tag": {
"rules": "rules",
"#tag": "rules.[1].name",
"#(2,tag)": "rules.[1].values[0]"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"userBias?": "=toInteger"
}
}
]
Which works fine and produces the following JSON:
{
"item": "television",
"userBias": 2,
"user": "123456",
"rules": [
{
"name": "category",
"values": [
"electronics"
]
},
{
"name": "tag",
"values": [
"summer"
]
}
]
}
If from the input though I delete "category": "electronics" so it becomes:
{
"user": "123456",
"product": "television",
"tag": "summer"
}
Then i get back the following result:
{
"item": "television",
"userBias": 2,
"user": "123456",
"rules": [
null,
{
"name": "tag",
"values": [
"summer"
]
}
]
}
The problem with the above is that it contains a null element inside the array and I do not know how to get rid of it. I have tried with recursivelySquashNulls but it does not work.
Also basically what am looking for is if both category and tag exist then tag should go to rules[1] if only tag exists then tag should go to rules[0].
Thanks in advance,
giannis
Because you specify tag and category elements individually. Rather, prefer putting them into the category rest by combining them under asterisked key notation such as
[
{
"operation": "shift",
"spec": {
"product": {
"#(1,product)": "item",
"#(1,user)": {
"#2": "userBias"
}
},
"user": {
"#(1,user)": "user"
},
"*": {
"$": "r[0].&.name",
"#(1,&)": "r[0].&.values[]"
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"r": {
"*": { "*": "rules" }
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"userBias?": "=toInteger"
}
}
]
Result1 :
Result2(without "category": "electronics" pair for the Input) :
I would like to move the recommendations array by row_id key inside the investors with the same row_id
Original Json
{
"investors": [
{
"row_id": 1,
"name": "AAAA"
},
{
"row_id": 2,
"name": "BBBB"
}
],
"recommendations": [
{
"row_id": "1",
"title": "ABC"
},
{
"row_id": "2",
"title": "CDE"
}
]
}
I've tried a lot of specs at https://jolt-demo.appspot.com with no success
Specs tried...
[{
"operation": "shift",
"spec": {
"investors": {
"*": "investors[]"
},
"recommendations": {
"#": "recommendations[]"
}
}
}]
Desired Json
{
"investors": [
{
"row_id": 1,
"name": "AAAA",
"recommendations":[{
"row_id": "1",
"title": "ABC"
}]
},
{
"row_id": 2,
"name": "BBBB",
"recommendations":[{
"row_id": "2",
"title": "CDE"
}]
}
]
}
This can be done in two stage shift
First shift groups everything based on row_id.
(I'd suggest running the first shift of its own to see what the output is)
Second shift uses that grouped output and formats results.
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"row_id": {
"*": {
"#2": "&.&4"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"investors": "investors.[#2]",
"recommendations": "investors.[#2].recommendations[]"
}
}
}
]
My input JSON looks like below, but i am not sure of how to do internal array related parameter transformation using JOLT. Any help is appreciated as i am new to JOLT
{
"pktId": 7603,
"seq": 1,
"vehicleNumber": "66079",
"rmdLocation": "1",
"rmdTime": "2019-01-07T11:27:05.745Z",
"position": {
"lat": 55.4911232,
"lng": -3.686831
},
"dataSource": 11,
"frames": [
{
"seq": 0,
"card": 8,
"channel": 6,
"value": 117
},
{
"seq": 1,
"card": 8,
"channel": 6,
"value": 120
}
]
}
Below is the spec file i have created but it is not complete
[
{
"operation": "shift",
"spec": {
"frames": {
"*": {
"seq": "parameters[&1].seq",
"card": "parameters[&1].card",
"channel": "parameters[&1].channel",
"value": "parameters[&1].value"
}
},
"rmdTime": "messageTime",
"vehicleNumber": "roadNumber"
}
},
{
"operation": "default",
"spec": {
"appId": "configMsgXX",
"customerId": "ABC",
"messageRev": 1,
"messageType": "customStatistics"
}
}
]
Expected output is as below
{
"appId": "configMsgXX",
"customerId": "ABC",
"deviceId": string1+roadNumber+string2",
"messageRev": 1,
"messageTime": 1543395341000,
"messageType": "customStatistics",
"parameters": [
{
"address": string1+string2,
"name": "EM2000VoltageMainGenerator",
"timestamp": 1543395341000,
"quality": "3",
"datatype": "INTEGER",
"value": 100,
"qualityReason": "Stale Data",
"category": "REAL"
}
],
"roadNumber": 66079
}
I am using this library https://github.com/bazaarvoice/jolt
[
{
"operation": "shift",
"spec": {
"frames": {
"*": {
"seq": "parameters[&1].seq",
"card": "parameters[&1].card",
"channel": "parameters[&1].channel",
"value": "parameters[&1].value",
"#(2,rmdTime)": "parameters[&1].timestamp"
}
},
"rmdTime": "messageTime",
"vehicleNumber": "roadNumber"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"parameters": {
"*": {
"quality": "3",
"name": "",
"address": "",
"datatype": "INTEGER",
"qualityReason": "Stale Data",
"category": "REAL"
}
}
}
},
{
"operation": "default",
"spec": {
"appId": "configMsgXX",
"customerId": "ABC",
"messageRev": 1,
"messageType": "customStatistics"
}
}
]
I have a complex JSON object (I've simplified it for this example) that I cannot figure out the JOLT transform JSON for. Does anybody have any ideas of what the JOLT spec file should be?
Original JSON
[
{
"date": {
"isoDate": "2019-03-22"
},
"application": {
"name": "SiebelProject"
},
"applicationResults": [
{
"reference": {
"name": "Number of Code Lines"
},
"result": {
"value": 44501
}
},
{
"reference": {
"name": "Transferability"
},
"result": {
"grade": 3.1889542208002064
}
}
]
},
{
"date": {
"isoDate": "2019-03-21"
},
"application": {
"name": "SiebelProject"
},
"applicationResults": [
{
"reference": {
"name": "Number of Code Lines"
},
"result": {
"value": 45000
}
},
{
"reference": {
"name": "Transferability"
},
"result": {
"grade": 3.8
}
}
]
}
]
Desired JSON after transformation and sorting by "Name" ASC, "Date" DESC
[
{
"Name": "SiebelProject",
"Date": "2019-03-22",
"Number of Code Lines": 44501,
"Transferability" : 3.1889542208002064
},
{
"Name": "SiebelProject",
"Date": "2019-03-21",
"Number of Code Lines": 45000,
"Transferability" : 3.8
}
]
I couldn't find a way to do the sort (I'm not even sure you can sort descending in JOLT) but here's a spec to do the transform:
[
{
"operation": "shift",
"spec": {
"*": {
"date": {
"isoDate": "[#3].Date"
},
"application": {
"name": "[#3].Name"
},
"applicationResults": {
"*": {
"reference": {
"name": {
"Number of Code Lines": {
"#(3,result.value)": "[#7].Number of Code Lines"
},
"Transferability": {
"#(3,result.grade)": "[#7].Transferability"
}
}
}
}
}
}
}
}
]
After that there are some tools (like jq I think) that could do the sort.
I am having a tough time using Jolt mapping trying to transform the input to the necessary form
Input JSON
{
"data": [
{
"name": "Alcohol",
"collection_id": 123,
"properties": [
{
"name": "Tax",
"property_id": "00001"
},
{
"name": "Expenditure",
"property_id": "00002"
}
],
"attributes": [
{
"name": "alcohol_tax",
"attribute_id": "00011"
},
{
"name": "alcohol_expenditure",
"attribute_id": "00022"
}
]
}
]
}
Output JSON
[
{
"name": "Alcohol",
"collection_id": 123,
"details": [{
"property_name": "Tax",
"property_id": "00001",
"attribute_id": "00011"
},
{
"property_name": "Expenditure",
"property_id": "00002",
"attribute_id": "00022"
}
]
}
]
I have tried a couple of ways to combine the arrays using a few rules but with little success.
One of the rules
[{
"operation": "shift",
"spec": {
"data": {
"*": {
"name": "&1.name",
"collection_id": "&1.collection_id",
"attributes": {
"*": {
"attribute_id": "&1.attribute_id[]"
}
},
"properties": {
"*": {
"name": "&1.myname[]",
"property_id": "&1.property_id[]"
}
}
}
}
}
}]
is adding all the attributes and properties to all the collections.I do not know why this happens as I thought &1.property_id[] would only add items in that particular collection to the array and not all collections. Any help/clues on why this is happening would be truly appreciated.
See solution below:
[0] creates the wrapping array
[&1] uses the position of the respective arrays so the results are combined in details the important part is wrapping square brackets so its treated as array rather than literal.
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"name": "[0].name",
"collection_id": "[0].collection_id",
"attributes": {
"*": {
"attribute_id": "[0].details.[&1].attribute_id"
}
},
"properties": {
"*": {
"name": "[0].details.[&1].name",
"property_id": "[0].details.[&1].property_id"
}
}
}
}
}
}
]
Produces the following:
[
{
"name": "Alcohol",
"collection_id": 123,
"details": [
{
"attribute_id": "00011",
"name": "Tax",
"property_id": "00001"
},
{
"attribute_id": "00022",
"name": "Expenditure",
"property_id": "00002"
}
]
}
]