Jolt spec convert 2d array to 1d array - jolt

How to convert 2d array to 1d array
{ "test": [ ["1", "2", "3"], ["4"] ] }
to
{ "test" : ["1", "2", "3", "4"] }
I tried multiple things and it didn't work

This spec should do the trick:
[
{
"operation": "shift",
"spec": {
"test": {
"*": {
"*": "test"
}
}
}
}
]

Related

Facing issue with JOLT transformation with nested array

I have a JSON input :
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"characteristic": [
{
"name": "char1",
"value": "PRE1"
},
{
"name": "char2",
"value": "2050-01-01"
}
]
},
{
"id": "ID_2",
"characteristic": [
{
"name": "char1",
"value": "PRE2"
},
{
"name": "char2",
"value": "2050-01-02"
}
]
}
]
}
which needs to be converted by using a Jolt transformation spec to the following output :
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"char1": "PRE1",
"char2": "2050-01-01"
},
{
"id": "ID_2",
"char1": "PRE2",
"char2": "2050-01-02"
}
]
}
Currently, I'm using this spec :
[
{
"operation": "shift",
"spec": {
"id": "id",
"Item": {
"*": {
"characteristic": {
"*": {
"name": {
"char1": {
"#(2,value)": "item[#3].char1"
},
"char2": {
"#(2,value)": "item[#3].char2"
}
}
}
}
}
}
}
}
]
which does not produce the desired result.
Can you please help me prepare a correct spec to handle this issue ?
Edit : What if I'd like to get the following JSON result ?
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"char1": "PRE1"
},
{
"id": "ID_2",
"char1": "PRE2",
"char2": "2050-01-02"
}
]
}
You can use the following shift transformation spec in which #value and #name are matched reciprocally such as
[
{
"operation": "shift",
"spec": {
"id": "&",
"Item": {
"*": {
"id": "&2[&1].&",// you can reduce two levels based on the inner identifier 4,3 -> 2,1
"char*": {
"*": {
"#value": "&4[&3].#name" // &4 copies the literal "Item" after going four levels up the tree, [&3] generates array-wise result(array of objects) after going three levels up the tree to reach the level of the indexes of the "Item" array
}
}
}
}
}
}
]
Edit : You can alternatively use the following spec for the case in which you need to return char2 within the all objects but the first :
[
{
"operation": "shift",
"spec": {
"id": "&",
"Item": {
"0": {// stands for the first index
"id": "&2[&1].&",
"char*": {
"*": {
"name": {
"char1": {
"#2,value": "&6[&5].#3,name"
}
}
}
}
},
"*": {
"id": "&2[&1].&",
"char*": {
"*": {
"#value": "&4[&3].#name"
}
}
}
}
}
}
]

Merge similar JSON messages into one using jolt

some help required for a jolt transformation that I am trying to achieve. Mainly I need to retain the key_id and contacts element and then group all elements in contacts into an array. This to merge multiple JSON messages into one batch json group
I have this input
[
{
"key_id": "436",
"contacts": [
{
"436": "aaaa-10f2-4cc3-820c-444444444",
"1378": "2021-11-08",
"1381": "2",
"1421": "1"
}
]
},
{
"key_id": "436",
"contacts": [
{
"436": "1111111111-e213-4c90-a8e8-6666gtggggf",
"1378": "2021-11-09",
"1381": "2",
"1421": "1"
}
]
},
{
"key_id": "436",
"contacts": [
{
"436": "xxxxxxxx-e213-4c90-a8e8-xxxxxxxxxxx",
"1378": "2021-11-05",
"1381": "2",
"1421": "1"
}
]
}
]
And would like to transform it with jolt into something like this
[
{
"key_id": "436",
"contacts": [
{
"436": "aaaa-10f2-4cc3-820c-444444444",
"1378": "2021-11-08",
"1381": "2",
"1421": "1"
},
{
"436": "1111111111-e213-4c90-a8e8-6666gtggggf",
"1378": "2021-11-09",
"1381": "2",
"1421": "1"
},
{
"436": "xxxxxxxx-e213-4c90-a8e8-xxxxxxxxxxx",
"1378": "2021-11-05",
"1381": "2",
"1421": "1"
}
]
}
]
I've tried multiple transformations, but no luck so far. Thanks in advance
You can use shift transformation spec in order to combine the sub-objects of the
contacts array respectively while retain all identical elements of key_id, and then pick leftmost key_id through use of cardinality transformation such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": "[0].&",
"contacts": {
"*": "[0].&1"
}
}
}
},
{
"operation": "cardinality",
"spec": {
"*": {
"key_id": "ONE"
}
}
}
]

JOLT to join different properties

Source JSON:
[
{
"metrics": {
"cost_per_app_install": "0.08",
},
"dimensions": {
"stat_time_day": "2021-06-04 00:00:00",
"campaign_id": 170011516
}
},
{
"metrics": {
"cost_per_app_install": "12.3",
},
"dimensions": {
"stat_time_day": "2021-06-04 00:00:00",
"campaign_id": 17013
}
}
]
Expected:
[
{
"cost_per_app_install": "0.08",
"stat_time_day": "2021-06-04 00:00:00",
"campaign_id": 170011516
},
{
"cost_per_app_install": "12.3",
"stat_time_day": "2021-06-04 00:00:00",
"campaign_id": 17013
}
]
JOLT spec:
[
{
"operation": "shift",
"spec": {
"*": {
"metrics": "[&1]",
"dimensions": "[&1]"
}
}
}
]
I want to add data from dimensions and metrics properties as one combined record. But unable with my JOLT config. This config just removes metrics and dimensions namings. Can I do it only with shift operation?
You need one more step to roam the indexes such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": { "*": "[&2].&" }
}
}
}
]
Btw, no need to explicitly list the key-value pairs for the current case, just "*": "[&1]" should be preferred rather than "metrics": "[&1]","dimensions": "[&1]" for the current pairs if it were the case.

Add array inside object with same key in jolt spec

I would like to move the recommendations array by row_id key inside the investors with the same row_id
Original Json
{
"investors": [
{
"row_id": 1,
"name": "AAAA"
},
{
"row_id": 2,
"name": "BBBB"
}
],
"recommendations": [
{
"row_id": "1",
"title": "ABC"
},
{
"row_id": "2",
"title": "CDE"
}
]
}
I've tried a lot of specs at https://jolt-demo.appspot.com with no success
Specs tried...
[{
"operation": "shift",
"spec": {
"investors": {
"*": "investors[]"
},
"recommendations": {
"#": "recommendations[]"
}
}
}]
Desired Json
{
"investors": [
{
"row_id": 1,
"name": "AAAA",
"recommendations":[{
"row_id": "1",
"title": "ABC"
}]
},
{
"row_id": 2,
"name": "BBBB",
"recommendations":[{
"row_id": "2",
"title": "CDE"
}]
}
]
}
This can be done in two stage shift
First shift groups everything based on row_id.
(I'd suggest running the first shift of its own to see what the output is)
Second shift uses that grouped output and formats results.
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"row_id": {
"*": {
"#2": "&.&4"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"investors": "investors.[#2]",
"recommendations": "investors.[#2].recommendations[]"
}
}
}
]

Issue while transforming the json using jolt spec

I am trying to jolt transformation spec for the below input :
{
"a": {
"serviceTesta": "Testa1",
"b": {
"serviceTestb": "Testb1",
"c": [
{
"name": "x",
"value": "100",
"desc": "this is description of X"
},
{
"name": "y",
"value": "200",
"desc": "this is description of y"
}
]
}
}
}
and the expected output is as below :
[
{
"Testa1_Testb1_x": "100",
"Testa1_Testb1_x_desc": "this is description of X"
},
{
"Testa1_Testb1_y": "100",
"Testa1_Testb1_y_desc": "this is description of y"
}
]
My Spec :
[
{
"operation": "shift",
"spec": {
"a": {
"b": {
"c": {
"*": {
"value": "#(4,serviceTesta).#(3,serviceTestb)#(1,name)"
}
}
}
}
}
}
]
I am new to jolt , tried different ways , but couldnt get desired output.
Any help is highly appreciated.
Thank you
You can achieve the desired result by doing secondary shift on your transformation:
Which uses the a combination of * and &n to build up your key.
[
{
"operation": "shift",
"spec": {
"a": {
"b": {
"c": {
"*": {
"value": "#(4,serviceTesta).#(3,serviceTestb).#(1,name)"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"#": "&3_&2_&1"
}
}
}
}
}
]