I am new to JOLT transformation. I am trying to create a transform spec.
I have an list of categories in the object where I need to only transform few details.
My sample code and spec re shown below.
In "0/SYS_CATALOG_DESCRIPTION" list, I need to convert it to a String based on the lang, i.e for en_US, I need to get AA Products
end result will be "_description" : "AA Products"
The "subCategories" should give me the following result:
"subCategories": [
{
"_id": "ce_155584",
"_parentIds": ["ce_128375"],
"_description": "Filters" //based on lang = en_US
}
]
Sample JSON:
{
"total": 16,
"max_score": 2.2809339,
"hits": [
{
"_index": "bosch-dms-frontend-service_en_us_1558584002",
"_type": "categories",
"_id": "ce_128375",
"_score": 2.2809339,
"_source": {
"_parentIds": [
"1234"
],
"0/SYS_CATALOG_DESCRIPTION": [
{
"lang": "de_DE",
"value": "AA Produkte"
},
{
"lang": "en_US",
"value": "AA Products"
}
],
"subCategories": [
{
"_index": "bosch-dms-frontend-service_en_us_1558584002",
"_type": "categories",
"_id": "ce_155584",
"_score": 2.2809339,
"_source": {
"_parentIds": [
"ce_128375"
],
"0/SYS_CATALOG_DESCRIPTION": [
{
"lang": "en_US",
"value": "Filters"
},
{
"lang": "zh_CN",
"value": "AA Filters (CN)"
}
],
"0/SYS_SYSTEMNAME": "AA_Filters"
}
}
]
}
}
]
}
SPEC:
[
{
"operation": "shift", // shift operation
"spec": {
"hits": {
"*": {
"_id": "_id",
"_source": {
"_parentIds": "_parentIds",
"0/SYS_CATALOG_DESCRIPTION": "_description",
}
}
}
}
}
]
The end result will be
{
"_id" : "ce_128375",
"_parentIds" : [ "1234" ],
"_description" : "AA Products (BR)",
"subCategories": [
{
"_id": "ce_155584",
"_score": 2.2809339,
"_parentIds": ["ce_128375"],
"_description" : "Filters"
}
]
}
I tried several ways but could not achieve the result.
Thank you.
Check if this spec is what you need:
[
{
"operation": "shift", // shift operation
"spec": {
"hits": {
"*": {
"_id": ["&",
"subCategories.[]._parentIds[]"],
"_source": {
"_parentIds": "&",
"0/SYS_CATALOG_DESCRIPTION": {
"*": {
"lang": {
"en_US": {
"#(2,value)": "_description"
}
}
}
},
"subCategories": {
"*": {
"_id": "subCategories.[&1].&",
"_score": "subCategories.[&1].&",
"_source": {
"0/SYS_CATALOG_DESCRIPTION": {
"*": {
"lang": {
"en_US": {
"#(2,value)": "subCategories.[&6]._description"
}
}
}
}
}
}
}
}
}
}
}
}
]
Related
I have a JSON input :
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"characteristic": [
{
"name": "char1",
"value": "PRE1"
},
{
"name": "char2",
"value": "2050-01-01"
}
]
},
{
"id": "ID_2",
"characteristic": [
{
"name": "char1",
"value": "PRE2"
},
{
"name": "char2",
"value": "2050-01-02"
}
]
}
]
}
which needs to be converted by using a Jolt transformation spec to the following output :
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"char1": "PRE1",
"char2": "2050-01-01"
},
{
"id": "ID_2",
"char1": "PRE2",
"char2": "2050-01-02"
}
]
}
Currently, I'm using this spec :
[
{
"operation": "shift",
"spec": {
"id": "id",
"Item": {
"*": {
"characteristic": {
"*": {
"name": {
"char1": {
"#(2,value)": "item[#3].char1"
},
"char2": {
"#(2,value)": "item[#3].char2"
}
}
}
}
}
}
}
}
]
which does not produce the desired result.
Can you please help me prepare a correct spec to handle this issue ?
Edit : What if I'd like to get the following JSON result ?
{
"id": "Root_ID",
"Item": [
{
"id": "ID_1",
"char1": "PRE1"
},
{
"id": "ID_2",
"char1": "PRE2",
"char2": "2050-01-02"
}
]
}
You can use the following shift transformation spec in which #value and #name are matched reciprocally such as
[
{
"operation": "shift",
"spec": {
"id": "&",
"Item": {
"*": {
"id": "&2[&1].&",// you can reduce two levels based on the inner identifier 4,3 -> 2,1
"char*": {
"*": {
"#value": "&4[&3].#name" // &4 copies the literal "Item" after going four levels up the tree, [&3] generates array-wise result(array of objects) after going three levels up the tree to reach the level of the indexes of the "Item" array
}
}
}
}
}
}
]
Edit : You can alternatively use the following spec for the case in which you need to return char2 within the all objects but the first :
[
{
"operation": "shift",
"spec": {
"id": "&",
"Item": {
"0": {// stands for the first index
"id": "&2[&1].&",
"char*": {
"*": {
"name": {
"char1": {
"#2,value": "&6[&5].#3,name"
}
}
}
}
},
"*": {
"id": "&2[&1].&",
"char*": {
"*": {
"#value": "&4[&3].#name"
}
}
}
}
}
}
]
I have this input JSON:
{
"user": "123456",
"product": "television",
"category": "electronics",
"tag": "summer"
}
And this transformation:
[
{
"operation": "shift",
"spec": {
"product": {
"#(1,product)": "item",
"#(1,user)": {
"#2": "userBias"
}
},
"user": {
"#(1,user)": "user"
},
"category": {
"#category": "rules.[0].name",
"#(1,category)": "rules.[0].values[0]"
},
"tag": {
"rules": "rules",
"#tag": "rules.[1].name",
"#(2,tag)": "rules.[1].values[0]"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"userBias?": "=toInteger"
}
}
]
Which works fine and produces the following JSON:
{
"item": "television",
"userBias": 2,
"user": "123456",
"rules": [
{
"name": "category",
"values": [
"electronics"
]
},
{
"name": "tag",
"values": [
"summer"
]
}
]
}
If from the input though I delete "category": "electronics" so it becomes:
{
"user": "123456",
"product": "television",
"tag": "summer"
}
Then i get back the following result:
{
"item": "television",
"userBias": 2,
"user": "123456",
"rules": [
null,
{
"name": "tag",
"values": [
"summer"
]
}
]
}
The problem with the above is that it contains a null element inside the array and I do not know how to get rid of it. I have tried with recursivelySquashNulls but it does not work.
Also basically what am looking for is if both category and tag exist then tag should go to rules[1] if only tag exists then tag should go to rules[0].
Thanks in advance,
giannis
Because you specify tag and category elements individually. Rather, prefer putting them into the category rest by combining them under asterisked key notation such as
[
{
"operation": "shift",
"spec": {
"product": {
"#(1,product)": "item",
"#(1,user)": {
"#2": "userBias"
}
},
"user": {
"#(1,user)": "user"
},
"*": {
"$": "r[0].&.name",
"#(1,&)": "r[0].&.values[]"
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"r": {
"*": { "*": "rules" }
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"userBias?": "=toInteger"
}
}
]
Result1 :
Result2(without "category": "electronics" pair for the Input) :
how can I do this?
This is the array....
Can you please help me?
Can you please give me the answer???? Thanks a lot
{
"results": {
"data": [
{
"name": "xx",
"typeRelationship": [
{
"relationship": "parent",
"type": {
"id": "yyyyy",
}
}
],
"id": "xxxxxxxx"
},
{
"name": "yy",
"typeRelationship": [
{
"relationshipType": "parent",
"type": {
"id": "CCCC"
}
},
{
"relationshipType": "child",
"service": {
"id": "DDDD"
}
},
{
"relationshipType": "child",
"service": {
"id": "xxxxxxxx"
}
}
],
"id": "yyyyy"
}
]
}}
expected:
This is expected:
{
"data" : [ {
"id" : "xxxx",
"href" : "xxxxxx",
"relation":"parent"
} ]
}
For some reason I need to type so it does let me update!!!
This works.
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"type": {
"id": {
"xxxx": {
"#3": "data[]"
}
}
}
}
}
}
}
]
Edit 1
The below spec moves all the values which as id=xxxxx to the data array.
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"type": {
"*": {
"id": {
"xxxx": {
"#(2)": "data[]",
"#(4,relation)": "data[&3].relation"
}
}
}
}
}
}
}
}
]
This totally works.
Thanks.
Can you please let me know what is 2? 3? 4?
Because my array is a bit different and I want to fix those numbers but does not work....
{
"results": {
"data": [
{
"name": "xx",
"typeRelationship": [
{
"relationship": "parent",
"type": {
"id": "yyyyy",
}
}
],
"id": "xxxxxxxx"
},
{
"name": "yy",
"typeRelationship": [
{
"relationshipType": "parent",
"type": {
"id": "CCCC"
}
},
{
"relationshipType": "child",
"service": {
"id": "DDDD"
}
},
{
"relationshipType": "child",
"service": {
"id": "xxxxxxxx"
}
}
],
"id": "yyyyy"
}
]
}
}
expected:
{
"rows" : [ {
"rowdata" : {
"relationshipType" : "child",
"Name" : "yy",
"id" : "yyyyy"
}
} ]
}
Im using mongoose, I have the following data of user collection:
[{
"_id": "1",
"notes": [
{
"value": "A90",
"text": "math"
},
{
"value": "A80",
"text": "english"
},
{
"value": "A70",
"text": "art"
}
]
},
{
"_id": "2",
"notes": [
{
"value": "A90",
"text": "math"
},
{
"value": "A80",
"text": "english"
}
]
},
{
"_id": "3",
"notes": [
{
"value": "A80",
"text": "art"
}
]
}]
and I have as a parameters the following array: [ "A90", "A80" ]
so I want to make a query to use this array to return only the records that have all the array items in the notes (value) table.
So for the example above it will return:
[{
"_id": "1",
"notes": [
{
"value": "A90",
"text": "math"
},
{
"value": "A80",
"text": "english"
},
{
"value": "A70",
"text": "art"
}
]
},
{
"_id": "2",
"notes": [
{
"value": "A90",
"text": "math"
},
{
"value": "A80",
"text": "english"
}
]
}]
I tried the following find query:
{ "notes": { $elemMatch: { value: { $in: valuesArray } } }}
but it returns a record even if just one element in valuesArray exist.
it turned out to be quite easy:
find({ "notes.value": { $all: arrayValues } })
I have a complex JSON object (I've simplified it for this example) that I cannot figure out the JOLT transform JSON for. Does anybody have any ideas of what the JOLT spec file should be?
Original JSON
[
{
"date": {
"isoDate": "2019-03-22"
},
"application": {
"name": "SiebelProject"
},
"applicationResults": [
{
"reference": {
"name": "Number of Code Lines"
},
"result": {
"value": 44501
}
},
{
"reference": {
"name": "Transferability"
},
"result": {
"grade": 3.1889542208002064
}
}
]
},
{
"date": {
"isoDate": "2019-03-21"
},
"application": {
"name": "SiebelProject"
},
"applicationResults": [
{
"reference": {
"name": "Number of Code Lines"
},
"result": {
"value": 45000
}
},
{
"reference": {
"name": "Transferability"
},
"result": {
"grade": 3.8
}
}
]
}
]
Desired JSON after transformation and sorting by "Name" ASC, "Date" DESC
[
{
"Name": "SiebelProject",
"Date": "2019-03-22",
"Number of Code Lines": 44501,
"Transferability" : 3.1889542208002064
},
{
"Name": "SiebelProject",
"Date": "2019-03-21",
"Number of Code Lines": 45000,
"Transferability" : 3.8
}
]
I couldn't find a way to do the sort (I'm not even sure you can sort descending in JOLT) but here's a spec to do the transform:
[
{
"operation": "shift",
"spec": {
"*": {
"date": {
"isoDate": "[#3].Date"
},
"application": {
"name": "[#3].Name"
},
"applicationResults": {
"*": {
"reference": {
"name": {
"Number of Code Lines": {
"#(3,result.value)": "[#7].Number of Code Lines"
},
"Transferability": {
"#(3,result.grade)": "[#7].Transferability"
}
}
}
}
}
}
}
}
]
After that there are some tools (like jq I think) that could do the sort.