Need JOLT spec file for transfer of complex JSON - jolt

I have a complex JSON object (I've simplified it for this example) that I cannot figure out the JOLT transform JSON for. Does anybody have any ideas of what the JOLT spec file should be?
Original JSON
[
{
"date": {
"isoDate": "2019-03-22"
},
"application": {
"name": "SiebelProject"
},
"applicationResults": [
{
"reference": {
"name": "Number of Code Lines"
},
"result": {
"value": 44501
}
},
{
"reference": {
"name": "Transferability"
},
"result": {
"grade": 3.1889542208002064
}
}
]
},
{
"date": {
"isoDate": "2019-03-21"
},
"application": {
"name": "SiebelProject"
},
"applicationResults": [
{
"reference": {
"name": "Number of Code Lines"
},
"result": {
"value": 45000
}
},
{
"reference": {
"name": "Transferability"
},
"result": {
"grade": 3.8
}
}
]
}
]
Desired JSON after transformation and sorting by "Name" ASC, "Date" DESC
[
{
"Name": "SiebelProject",
"Date": "2019-03-22",
"Number of Code Lines": 44501,
"Transferability" : 3.1889542208002064
},
{
"Name": "SiebelProject",
"Date": "2019-03-21",
"Number of Code Lines": 45000,
"Transferability" : 3.8
}
]

I couldn't find a way to do the sort (I'm not even sure you can sort descending in JOLT) but here's a spec to do the transform:
[
{
"operation": "shift",
"spec": {
"*": {
"date": {
"isoDate": "[#3].Date"
},
"application": {
"name": "[#3].Name"
},
"applicationResults": {
"*": {
"reference": {
"name": {
"Number of Code Lines": {
"#(3,result.value)": "[#7].Number of Code Lines"
},
"Transferability": {
"#(3,result.grade)": "[#7].Transferability"
}
}
}
}
}
}
}
}
]
After that there are some tools (like jq I think) that could do the sort.

Related

Facing issue with JOLT transformation with nested array

I have a JSON input :
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"characteristic": [
{
"name": "char1",
"value": "PRE1"
},
{
"name": "char2",
"value": "2050-01-01"
}
]
},
{
"id": "ID_2",
"characteristic": [
{
"name": "char1",
"value": "PRE2"
},
{
"name": "char2",
"value": "2050-01-02"
}
]
}
]
}
which needs to be converted by using a Jolt transformation spec to the following output :
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"char1": "PRE1",
"char2": "2050-01-01"
},
{
"id": "ID_2",
"char1": "PRE2",
"char2": "2050-01-02"
}
]
}
Currently, I'm using this spec :
[
{
"operation": "shift",
"spec": {
"id": "id",
"Item": {
"*": {
"characteristic": {
"*": {
"name": {
"char1": {
"#(2,value)": "item[#3].char1"
},
"char2": {
"#(2,value)": "item[#3].char2"
}
}
}
}
}
}
}
}
]
which does not produce the desired result.
Can you please help me prepare a correct spec to handle this issue ?
Edit : What if I'd like to get the following JSON result ?
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"char1": "PRE1"
},
{
"id": "ID_2",
"char1": "PRE2",
"char2": "2050-01-02"
}
]
}
You can use the following shift transformation spec in which #value and #name are matched reciprocally such as
[
{
"operation": "shift",
"spec": {
"id": "&",
"Item": {
"*": {
"id": "&2[&1].&",// you can reduce two levels based on the inner identifier 4,3 -> 2,1
"char*": {
"*": {
"#value": "&4[&3].#name" // &4 copies the literal "Item" after going four levels up the tree, [&3] generates array-wise result(array of objects) after going three levels up the tree to reach the level of the indexes of the "Item" array
}
}
}
}
}
}
]
Edit : You can alternatively use the following spec for the case in which you need to return char2 within the all objects but the first :
[
{
"operation": "shift",
"spec": {
"id": "&",
"Item": {
"0": {// stands for the first index
"id": "&2[&1].&",
"char*": {
"*": {
"name": {
"char1": {
"#2,value": "&6[&5].#3,name"
}
}
}
}
},
"*": {
"id": "&2[&1].&",
"char*": {
"*": {
"#value": "&4[&3].#name"
}
}
}
}
}
}
]

JOLT Nested if condition

I have this input JSON:
{
"user": "123456",
"product": "television",
"category": "electronics",
"tag": "summer"
}
And this transformation:
[
{
"operation": "shift",
"spec": {
"product": {
"#(1,product)": "item",
"#(1,user)": {
"#2": "userBias"
}
},
"user": {
"#(1,user)": "user"
},
"category": {
"#category": "rules.[0].name",
"#(1,category)": "rules.[0].values[0]"
},
"tag": {
"rules": "rules",
"#tag": "rules.[1].name",
"#(2,tag)": "rules.[1].values[0]"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"userBias?": "=toInteger"
}
}
]
Which works fine and produces the following JSON:
{
"item": "television",
"userBias": 2,
"user": "123456",
"rules": [
{
"name": "category",
"values": [
"electronics"
]
},
{
"name": "tag",
"values": [
"summer"
]
}
]
}
If from the input though I delete "category": "electronics" so it becomes:
{
"user": "123456",
"product": "television",
"tag": "summer"
}
Then i get back the following result:
{
"item": "television",
"userBias": 2,
"user": "123456",
"rules": [
null,
{
"name": "tag",
"values": [
"summer"
]
}
]
}
The problem with the above is that it contains a null element inside the array and I do not know how to get rid of it. I have tried with recursivelySquashNulls but it does not work.
Also basically what am looking for is if both category and tag exist then tag should go to rules[1] if only tag exists then tag should go to rules[0].
Thanks in advance,
giannis
Because you specify tag and category elements individually. Rather, prefer putting them into the category rest by combining them under asterisked key notation such as
[
{
"operation": "shift",
"spec": {
"product": {
"#(1,product)": "item",
"#(1,user)": {
"#2": "userBias"
}
},
"user": {
"#(1,user)": "user"
},
"*": {
"$": "r[0].&.name",
"#(1,&)": "r[0].&.values[]"
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"r": {
"*": { "*": "rules" }
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"userBias?": "=toInteger"
}
}
]
Result1 :
Result2(without "category": "electronics" pair for the Input) :

JOLT - filtering array based on object value

how can I do this?
This is the array....
Can you please help me?
Can you please give me the answer???? Thanks a lot
{
"results": {
"data": [
{
"name": "xx",
"typeRelationship": [
{
"relationship": "parent",
"type": {
"id": "yyyyy",
}
}
],
"id": "xxxxxxxx"
},
{
"name": "yy",
"typeRelationship": [
{
"relationshipType": "parent",
"type": {
"id": "CCCC"
}
},
{
"relationshipType": "child",
"service": {
"id": "DDDD"
}
},
{
"relationshipType": "child",
"service": {
"id": "xxxxxxxx"
}
}
],
"id": "yyyyy"
}
]
}}
expected:
This is expected:
{
"data" : [ {
"id" : "xxxx",
"href" : "xxxxxx",
"relation":"parent"
} ]
}
For some reason I need to type so it does let me update!!!
This works.
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"type": {
"id": {
"xxxx": {
"#3": "data[]"
}
}
}
}
}
}
}
]
Edit 1
The below spec moves all the values which as id=xxxxx to the data array.
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"type": {
"*": {
"id": {
"xxxx": {
"#(2)": "data[]",
"#(4,relation)": "data[&3].relation"
}
}
}
}
}
}
}
}
]
This totally works.
Thanks.
Can you please let me know what is 2? 3? 4?
Because my array is a bit different and I want to fix those numbers but does not work....
{
"results": {
"data": [
{
"name": "xx",
"typeRelationship": [
{
"relationship": "parent",
"type": {
"id": "yyyyy",
}
}
],
"id": "xxxxxxxx"
},
{
"name": "yy",
"typeRelationship": [
{
"relationshipType": "parent",
"type": {
"id": "CCCC"
}
},
{
"relationshipType": "child",
"service": {
"id": "DDDD"
}
},
{
"relationshipType": "child",
"service": {
"id": "xxxxxxxx"
}
}
],
"id": "yyyyy"
}
]
}
}
expected:
{
"rows" : [ {
"rowdata" : {
"relationshipType" : "child",
"Name" : "yy",
"id" : "yyyyy"
}
} ]
}

Transform list array inside an array in json

I am new to JOLT transformation. I am trying to create a transform spec.
I have an list of categories in the object where I need to only transform few details.
My sample code and spec re shown below.
In "0/SYS_CATALOG_DESCRIPTION" list, I need to convert it to a String based on the lang, i.e for en_US, I need to get AA Products
end result will be "_description" : "AA Products"
The "subCategories" should give me the following result:
"subCategories": [
{
"_id": "ce_155584",
"_parentIds": ["ce_128375"],
"_description": "Filters" //based on lang = en_US
}
]
Sample JSON:
{
"total": 16,
"max_score": 2.2809339,
"hits": [
{
"_index": "bosch-dms-frontend-service_en_us_1558584002",
"_type": "categories",
"_id": "ce_128375",
"_score": 2.2809339,
"_source": {
"_parentIds": [
"1234"
],
"0/SYS_CATALOG_DESCRIPTION": [
{
"lang": "de_DE",
"value": "AA Produkte"
},
{
"lang": "en_US",
"value": "AA Products"
}
],
"subCategories": [
{
"_index": "bosch-dms-frontend-service_en_us_1558584002",
"_type": "categories",
"_id": "ce_155584",
"_score": 2.2809339,
"_source": {
"_parentIds": [
"ce_128375"
],
"0/SYS_CATALOG_DESCRIPTION": [
{
"lang": "en_US",
"value": "Filters"
},
{
"lang": "zh_CN",
"value": "AA Filters (CN)"
}
],
"0/SYS_SYSTEMNAME": "AA_Filters"
}
}
]
}
}
]
}
SPEC:
[
{
"operation": "shift", // shift operation
"spec": {
"hits": {
"*": {
"_id": "_id",
"_source": {
"_parentIds": "_parentIds",
"0/SYS_CATALOG_DESCRIPTION": "_description",
}
}
}
}
}
]
The end result will be
{
"_id" : "ce_128375",
"_parentIds" : [ "1234" ],
"_description" : "AA Products (BR)",
"subCategories": [
{
"_id": "ce_155584",
"_score": 2.2809339,
"_parentIds": ["ce_128375"],
"_description" : "Filters"
}
]
}
I tried several ways but could not achieve the result.
Thank you.
Check if this spec is what you need:
[
{
"operation": "shift", // shift operation
"spec": {
"hits": {
"*": {
"_id": ["&",
"subCategories.[]._parentIds[]"],
"_source": {
"_parentIds": "&",
"0/SYS_CATALOG_DESCRIPTION": {
"*": {
"lang": {
"en_US": {
"#(2,value)": "_description"
}
}
}
},
"subCategories": {
"*": {
"_id": "subCategories.[&1].&",
"_score": "subCategories.[&1].&",
"_source": {
"0/SYS_CATALOG_DESCRIPTION": {
"*": {
"lang": {
"en_US": {
"#(2,value)": "subCategories.[&6]._description"
}
}
}
}
}
}
}
}
}
}
}
}
]

Combining two arrays and transformation using Jolt

I am having a tough time using Jolt mapping trying to transform the input to the necessary form
Input JSON
{
"data": [
{
"name": "Alcohol",
"collection_id": 123,
"properties": [
{
"name": "Tax",
"property_id": "00001"
},
{
"name": "Expenditure",
"property_id": "00002"
}
],
"attributes": [
{
"name": "alcohol_tax",
"attribute_id": "00011"
},
{
"name": "alcohol_expenditure",
"attribute_id": "00022"
}
]
}
]
}
Output JSON
[
{
"name": "Alcohol",
"collection_id": 123,
"details": [{
"property_name": "Tax",
"property_id": "00001",
"attribute_id": "00011"
},
{
"property_name": "Expenditure",
"property_id": "00002",
"attribute_id": "00022"
}
]
}
]
I have tried a couple of ways to combine the arrays using a few rules but with little success.
One of the rules
[{
"operation": "shift",
"spec": {
"data": {
"*": {
"name": "&1.name",
"collection_id": "&1.collection_id",
"attributes": {
"*": {
"attribute_id": "&1.attribute_id[]"
}
},
"properties": {
"*": {
"name": "&1.myname[]",
"property_id": "&1.property_id[]"
}
}
}
}
}
}]
is adding all the attributes and properties to all the collections.I do not know why this happens as I thought &1.property_id[] would only add items in that particular collection to the array and not all collections. Any help/clues on why this is happening would be truly appreciated.
See solution below:
[0] creates the wrapping array
[&1] uses the position of the respective arrays so the results are combined in details the important part is wrapping square brackets so its treated as array rather than literal.
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"name": "[0].name",
"collection_id": "[0].collection_id",
"attributes": {
"*": {
"attribute_id": "[0].details.[&1].attribute_id"
}
},
"properties": {
"*": {
"name": "[0].details.[&1].name",
"property_id": "[0].details.[&1].property_id"
}
}
}
}
}
}
]
Produces the following:
[
{
"name": "Alcohol",
"collection_id": 123,
"details": [
{
"attribute_id": "00011",
"name": "Tax",
"property_id": "00001"
},
{
"attribute_id": "00022",
"name": "Expenditure",
"property_id": "00002"
}
]
}
]