some help required for a jolt transformation that I am trying to achieve. Mainly I need to retain the key_id and contacts element and then group all elements in contacts into an array. This to merge multiple JSON messages into one batch json group
I have this input
[
{
"key_id": "436",
"contacts": [
{
"436": "aaaa-10f2-4cc3-820c-444444444",
"1378": "2021-11-08",
"1381": "2",
"1421": "1"
}
]
},
{
"key_id": "436",
"contacts": [
{
"436": "1111111111-e213-4c90-a8e8-6666gtggggf",
"1378": "2021-11-09",
"1381": "2",
"1421": "1"
}
]
},
{
"key_id": "436",
"contacts": [
{
"436": "xxxxxxxx-e213-4c90-a8e8-xxxxxxxxxxx",
"1378": "2021-11-05",
"1381": "2",
"1421": "1"
}
]
}
]
And would like to transform it with jolt into something like this
[
{
"key_id": "436",
"contacts": [
{
"436": "aaaa-10f2-4cc3-820c-444444444",
"1378": "2021-11-08",
"1381": "2",
"1421": "1"
},
{
"436": "1111111111-e213-4c90-a8e8-6666gtggggf",
"1378": "2021-11-09",
"1381": "2",
"1421": "1"
},
{
"436": "xxxxxxxx-e213-4c90-a8e8-xxxxxxxxxxx",
"1378": "2021-11-05",
"1381": "2",
"1421": "1"
}
]
}
]
I've tried multiple transformations, but no luck so far. Thanks in advance
You can use shift transformation spec in order to combine the sub-objects of the
contacts array respectively while retain all identical elements of key_id, and then pick leftmost key_id through use of cardinality transformation such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": "[0].&",
"contacts": {
"*": "[0].&1"
}
}
}
},
{
"operation": "cardinality",
"spec": {
"*": {
"key_id": "ONE"
}
}
}
]
Related
I have a JSON input :
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"characteristic": [
{
"name": "char1",
"value": "PRE1"
},
{
"name": "char2",
"value": "2050-01-01"
}
]
},
{
"id": "ID_2",
"characteristic": [
{
"name": "char1",
"value": "PRE2"
},
{
"name": "char2",
"value": "2050-01-02"
}
]
}
]
}
which needs to be converted by using a Jolt transformation spec to the following output :
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"char1": "PRE1",
"char2": "2050-01-01"
},
{
"id": "ID_2",
"char1": "PRE2",
"char2": "2050-01-02"
}
]
}
Currently, I'm using this spec :
[
{
"operation": "shift",
"spec": {
"id": "id",
"Item": {
"*": {
"characteristic": {
"*": {
"name": {
"char1": {
"#(2,value)": "item[#3].char1"
},
"char2": {
"#(2,value)": "item[#3].char2"
}
}
}
}
}
}
}
}
]
which does not produce the desired result.
Can you please help me prepare a correct spec to handle this issue ?
Edit : What if I'd like to get the following JSON result ?
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"char1": "PRE1"
},
{
"id": "ID_2",
"char1": "PRE2",
"char2": "2050-01-02"
}
]
}
You can use the following shift transformation spec in which #value and #name are matched reciprocally such as
[
{
"operation": "shift",
"spec": {
"id": "&",
"Item": {
"*": {
"id": "&2[&1].&",// you can reduce two levels based on the inner identifier 4,3 -> 2,1
"char*": {
"*": {
"#value": "&4[&3].#name" // &4 copies the literal "Item" after going four levels up the tree, [&3] generates array-wise result(array of objects) after going three levels up the tree to reach the level of the indexes of the "Item" array
}
}
}
}
}
}
]
Edit : You can alternatively use the following spec for the case in which you need to return char2 within the all objects but the first :
[
{
"operation": "shift",
"spec": {
"id": "&",
"Item": {
"0": {// stands for the first index
"id": "&2[&1].&",
"char*": {
"*": {
"name": {
"char1": {
"#2,value": "&6[&5].#3,name"
}
}
}
}
},
"*": {
"id": "&2[&1].&",
"char*": {
"*": {
"#value": "&4[&3].#name"
}
}
}
}
}
}
]
I am looking for some inputs to remove duplicates from a json string similar to below sample :
Sample Input :
{
"profile": true,
"address": [
"12",
"23"
],
"zipCodes": [
"12345",
"56789",
"12345",
"56789"
],
"phoneNumber": [
"87857",
"927465",
"274894",
"87857"
],
"userName": [
"ABC",
"PQR",
"ABC"
],
"enableEmailNot": "No"
}
Expected Output :
{
"profile": true,
"zipCodes": [
"12345",
"56789"
],
"phoneNumber": [
"87857",
"927465",
"274894"
],
"userName": [
"ABC",
"PQR"
],
"address": [
"12",
"23"
],
"enableEmailNot": "No"
}
Thanks for the help
You can use following specs
[
{
// exchange key-value pairs while seperating lists and non-lists by prefixing "xx" non-lists
"operation": "shift",
"spec": {
"profile": "xx.&",
"*": {
"*": {
"$": "&2.#(0)"
}
},
"enableEmailNot": "xx.&"
}
},
{
// pick the first components of lists
"operation": "cardinality",
"spec": {
"*": {
"*": "ONE"
}
}
},
{
// reform the lists through use of "$" operator
"operation": "shift",
"spec": {
"xx": {
"*": "&"
},
"*": {
"*": {
"$": "&2[]"
}
}
}
}
]
Source JSON:
[
{
"metrics": {
"cost_per_app_install": "0.08",
},
"dimensions": {
"stat_time_day": "2021-06-04 00:00:00",
"campaign_id": 170011516
}
},
{
"metrics": {
"cost_per_app_install": "12.3",
},
"dimensions": {
"stat_time_day": "2021-06-04 00:00:00",
"campaign_id": 17013
}
}
]
Expected:
[
{
"cost_per_app_install": "0.08",
"stat_time_day": "2021-06-04 00:00:00",
"campaign_id": 170011516
},
{
"cost_per_app_install": "12.3",
"stat_time_day": "2021-06-04 00:00:00",
"campaign_id": 17013
}
]
JOLT spec:
[
{
"operation": "shift",
"spec": {
"*": {
"metrics": "[&1]",
"dimensions": "[&1]"
}
}
}
]
I want to add data from dimensions and metrics properties as one combined record. But unable with my JOLT config. This config just removes metrics and dimensions namings. Can I do it only with shift operation?
You need one more step to roam the indexes such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": { "*": "[&2].&" }
}
}
}
]
Btw, no need to explicitly list the key-value pairs for the current case, just "*": "[&1]" should be preferred rather than "metrics": "[&1]","dimensions": "[&1]" for the current pairs if it were the case.
I have this input JSON:
{
"user": "123456",
"product": "television",
"category": "electronics",
"tag": "summer"
}
And this transformation:
[
{
"operation": "shift",
"spec": {
"product": {
"#(1,product)": "item",
"#(1,user)": {
"#2": "userBias"
}
},
"user": {
"#(1,user)": "user"
},
"category": {
"#category": "rules.[0].name",
"#(1,category)": "rules.[0].values[0]"
},
"tag": {
"rules": "rules",
"#tag": "rules.[1].name",
"#(2,tag)": "rules.[1].values[0]"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"userBias?": "=toInteger"
}
}
]
Which works fine and produces the following JSON:
{
"item": "television",
"userBias": 2,
"user": "123456",
"rules": [
{
"name": "category",
"values": [
"electronics"
]
},
{
"name": "tag",
"values": [
"summer"
]
}
]
}
If from the input though I delete "category": "electronics" so it becomes:
{
"user": "123456",
"product": "television",
"tag": "summer"
}
Then i get back the following result:
{
"item": "television",
"userBias": 2,
"user": "123456",
"rules": [
null,
{
"name": "tag",
"values": [
"summer"
]
}
]
}
The problem with the above is that it contains a null element inside the array and I do not know how to get rid of it. I have tried with recursivelySquashNulls but it does not work.
Also basically what am looking for is if both category and tag exist then tag should go to rules[1] if only tag exists then tag should go to rules[0].
Thanks in advance,
giannis
Because you specify tag and category elements individually. Rather, prefer putting them into the category rest by combining them under asterisked key notation such as
[
{
"operation": "shift",
"spec": {
"product": {
"#(1,product)": "item",
"#(1,user)": {
"#2": "userBias"
}
},
"user": {
"#(1,user)": "user"
},
"*": {
"$": "r[0].&.name",
"#(1,&)": "r[0].&.values[]"
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"r": {
"*": { "*": "rules" }
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"userBias?": "=toInteger"
}
}
]
Result1 :
Result2(without "category": "electronics" pair for the Input) :
I would like to move the recommendations array by row_id key inside the investors with the same row_id
Original Json
{
"investors": [
{
"row_id": 1,
"name": "AAAA"
},
{
"row_id": 2,
"name": "BBBB"
}
],
"recommendations": [
{
"row_id": "1",
"title": "ABC"
},
{
"row_id": "2",
"title": "CDE"
}
]
}
I've tried a lot of specs at https://jolt-demo.appspot.com with no success
Specs tried...
[{
"operation": "shift",
"spec": {
"investors": {
"*": "investors[]"
},
"recommendations": {
"#": "recommendations[]"
}
}
}]
Desired Json
{
"investors": [
{
"row_id": 1,
"name": "AAAA",
"recommendations":[{
"row_id": "1",
"title": "ABC"
}]
},
{
"row_id": 2,
"name": "BBBB",
"recommendations":[{
"row_id": "2",
"title": "CDE"
}]
}
]
}
This can be done in two stage shift
First shift groups everything based on row_id.
(I'd suggest running the first shift of its own to see what the output is)
Second shift uses that grouped output and formats results.
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"row_id": {
"*": {
"#2": "&.&4"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"investors": "investors.[#2]",
"recommendations": "investors.[#2].recommendations[]"
}
}
}
]