Standard format for specifying the mapping of fields - interface

We are trying to build a single interface which can be integrated with multiple systems.
Hence we need a way to specify the mappings of standard fields across different systems.
Example:
Interface1 has the following fields: A, B, C
Interface2 has the following fields: P, Q, R
We need to map fields: A -> Q, B -> P and C -> R (just an example)
We can simply create a JSON of such mappings and follow our own convention, but wanted to check if there are any standard formats / open source tools for specifying or simplifying the mapping of fields from source to destination?

Assuming you are trying to convert:
Response from interface 1 = { "A": "value1", "B": "value2", "C": "value3" }
to:
Response from interface 2 = { "Q": "value1", "P": "value2", "R": "value3" }
You can use any library that does JSON transformations such as this or this. For instance, you could define your transformation like this:
var template = {
Q: '$.A',
P: '$.B',
R: '$.C'
};
var input = { "A": "value1", "B": "value2", "C": "value3" };
var output = jsonPathObjectTransform(input, template);
// -> output = { "Q": "value1", "P": "value2", "R": "value3" }
Using Java with Jolt (I was unable to test it, but that's the general idea):
String input = "{\n" +
" \"A\": \"value1\",\n" +
" \"B\": \"value2\",\n" +
" \"C\": \"value3\"\n" +
"}"
String spec = "[\n" +
" {\n" +
" \"operation\": \"shift\",\n" +
" \"spec\": {\n" +
" \"A\": \"Q\",\n" +
" \"B\": \"P\",\n" +
" \"C\": \"R\"\n" +
" }\n" +
" }\n" +
"]\n"
Chainr chainr = JsonUtils.jsonToList(spec);
Object output = chainr.transform(input);
// -> output = { "Q": "value1", "P": "value2", "R": "value3" }

Related

Multiple switch statement with conditional inside

Is there a cleaner way of writing the conditional for case "1", "4", "7"? For this case I want to write to the string "b" only if the string value is not "4".
var b = ""
var c = ""
let s = "4"
switch s {
case "0", "6", "8":
c += "|_|"
case "1", "4", "7":
if s != "4" { b += " |" }
c += " |"
case "2":
c += "|_ "
case "3", "5", "9":
c += " _|"
default:
c += ""
}
This is a place you could use the fallthrough statement:
switch s {
case "0", "6", "8":
c += "|_|"
case "1", "7":
b += " |"
fallthrough
case "4":
c += " |"
case "2":
c += "|_ "
case "3", "5", "9":
c += " _|"
default:
break
}
Since you want "1" and "7" to execute both statements, give them their own case and then follow it with fallthough which allows it to fall though to the next case. Then "1", "4", and "7" will all execute the code in the case "4": case.
I think the whole idea of switch statement is that you don’t use ifs inside. You should probably just create a separate case for “4”. The amount of space saved by putting this conditional isn’t worth obscuring the code IMO.

mongo c-driver bcon $ne value no equal

I am trying to append a requirement to my BCON query where 'tribe_type' does not equal 'initial-public'.
My original code was and that worked:
query = BCON_NEW ("_id", BCON_OID(&oid));
When I add the second part, it compiles, however the mongo match fails.
query = BCON_NEW ("_id", BCON_OID(&oid),
"{",
"tribe_type",
"$ne",
"initial-public",
"}"
);
You have to specify the BCON type for UTF-8 strings.
Be careful with implict $and's
Be careful with nested documents and BCON.
query = BCON_NEW ("_id", BCON_OID(&oid),
"{",
"tribe_type",
"$ne",
"initial-public",
"}"
);
compiles into this command
{ "_id" : <an oid> }, { "tribe_type" : "$ne" }
which is obviously not what you want.
be explicit with the $and operation, correctly type the string as a UTF8 field, and make sure you capture the nested documents like this:
query = BCON_NEW (
"$and", "[", "{", "_id", BCON_OID(&oid), "}",
"{", "tribe_type", "{", "$ne", BCON_UTF8 ("initial-public"), "}", "}","]"
);
yields a query that looks like this
{ "$and" : [ { "_id" : <an oid> }, { "tribe_type" : { "$ne" : "initial-public" } } ] }
which is probably what you want.
query = BCON_NEW ("_id", BCON_OID(&oid),
"tribe_type",
"{",
"$ne", BCON_UTF8 ("initial-public"),
"}");

MongoDb - Change type from Int to Double

We have a collection that looks like this:
{
"_id" : "10571:6",
"v" : 261355,
"ts" : 4.88387e+008
}
Now, some of the "v" are ints, some are doubles. I want to change them all to doubles.
I've tried a few things but nothing works (v is an int32 for this record, I want to change it to a double):
db.getCollection('VehicleLastValues')
.find
(
{_id : "10572:6"}
)
.forEach
(
function (x)
{
temp = x.v * 1.0;
db.getCollection('VehicleLastValues').save(x);
}}
Things I've tried:
x.v = x.v * 1.1 / 1.1;
x.v = parseFloat (new String(x.v));
But I can't get it to be saved as a double...
By default all "numbers" are stored as "double" in MongoDB unless generally cast overwise.
Take the following samples:
db.sample.insert({ "a": 1 })
db.sample.insert({ "a": NumberLong(1) })
db.sample.insert({ "a": NumberInt(1) })
db.sample.insert({ "a": 1.223 })
This yields a collection like this:
{ "_id" : ObjectId("559bb1b4a23c8a3da73e0f76"), "a" : 1 }
{ "_id" : ObjectId("559bb1bba23c8a3da73e0f77"), "a" : NumberLong(1) }
{ "_id" : ObjectId("559bb29aa23c8a3da73e0f79"), "a" : 1 }
{ "_id" : ObjectId("559bb30fa23c8a3da73e0f7a"), "a" : 1.223 }
Despite the different constructor functions note how several of the data points there look much the same. The MongoDB shell itself doesn't always clearly distinquish between them, but there is a way you can tell.
There is of course the $type query operator, which allows selection of BSON Types.
So testing this with Type 1 - Which is "double":
> db.sample.find({ "a": { "$type": 1 } })
{ "_id" : ObjectId("559bb1b4a23c8a3da73e0f76"), "a" : 1 }
{ "_id" : ObjectId("559bb30fa23c8a3da73e0f7a"), "a" : 1.223 }
You see that both the first insert and the last are selected, but of course not the other two.
So now test for BSON Type 16 - which is a 32-bit integer
> db.sample.find({ "a": { "$type": 16 } })
{ "_id" : ObjectId("559bb29aa23c8a3da73e0f79"), "a" : 1 }
That was the "third" insertion which used the NumberInt() function in the shell. So that function and other serialization from your driver can set this specific BSON type.
And for the BSON Type 18 - which is 64-bit integer
> db.sample.find({ "a": { "$type": 18 } })
{ "_id" : ObjectId("559bb1bba23c8a3da73e0f77"), "a" : NumberLong(1) }
The "second" insertion which was contructed via NumberLong().
If you wanted to "weed out" things that were "not a double" then you would do:
db.sample.find({ "$or": [{ "a": { "$type": 16 } },{ "a": { "$type": 18 } }]})
Which are the only other valid numeric types other than "double" itself.
So to "convert" these in your collection, you can "Bulk" process like this:
var bulk = db.sample.initializeUnorderedBulkOp(),
count = 0;
db.sample.find({
"$or": [
{ "a": { "$type": 16 } },
{ "a": { "$type": 18 } }
]
}).forEach(function(doc) {
bulk.find({ "_id": doc._id })
.updateOne({
"$set": { "b": doc.a.valueOf() } ,
"$unset": { "a": 1 }
});
bulk.find({ "_id": doc._id })
.updateOne({ "$rename": { "b": "a" } });
count++;
if ( count % 1000 == 0 ) {
bulk.execute()
bulk = db.sample.initializeUnOrderedBulkOp();
}
})
if ( count % 1000 != 0 ) bulk.execute();
What that does is performed in three steps "in bulk":
Re-cast the value to a new field as a "double"
Remove the old field with the unwanted type
Rename the new field to the old field name
This is necessary since the BSON type information is "sticky" to the field element once created. So in order to "re-cast" you need to completely remove the old data which includes the original field assignment.
So that should explain how to "detect" and also "re-cast" unwanted types in your documents.

Distinct/Aggregation query Mongodb array, trim trailing space

I have a MongoDB collection which contains a colours array like :
myCollection :
{
_id : ...,
"colours" : [
{
"colourpercentage" : "42",
"colourname" : "Blue"
},
{
"colourpercentage" : "32",
"colourname" : "Red"
},
{
"colourpercentage" : "10",
"colourname" : "Green "
}
]
}
I would like to retrieve every distinct colourname of every entry of this collection, and be able to filter it with a search.
I tried with distinct but without success. I searched further and found that an aggregation could help me. For the moment I have :
db.getCollection('myCollection').aggregate([
{ "$match": { "colours.colourname": /Gre/ } }, # Gre is my search
{ "$unwind": "$colours" },
{ "$match": { "colours.colourname": /search/ } },
{ "$group": {
"_id": "$colours.colourname"
}}
])
It is working, but I get an array like :
{
"result" : [
{
"_id" : "Grey"
},
{
"_id" : "Light Green "
},
{
"_id" : "Light Green"
},
{
"_id" : "Green "
},
{
"_id" : "Green"
}
],
"ok" : 1.0000000000000000
}
And I would like to remove duplicate entries which have a space in the end and displays them like :
["Grey","Light Green","Green"]
One approach you could take is the Map-Reduce way even though the JavaScript interpreter driven mapReduce takes a bit longer than the aggregation framework but will work since you will be using some very useful native JavaScript functions that are lacking in the aggregation framework. For instance, in the map function you could use the trim() function to remove any trailing spaces in your colourname fields so that you can emit the "cleansed" keys.
The Map-Reduce operation would typically have the following map and reduce functions:
var map = function() {
if (!this.colours) return;
this.colours.forEach(function (c){
emit(c.colourname.trim(), 1)
});
};
var reduce = function(key, values) {
var count = 0;
for (index in values) {
count += values[index];
}
return count;
};
db.runCommand( { mapreduce : "myCollection", map : map , reduce : reduce , out : "map_reduce_result" } );
You can then query map_reduce_result collection with the regex to have the result:
var getDistinctKeys = function (doc) { return doc._id };
var result = db.map_reduce_result.find({ "_id": /Gre/ }).map(getDistinctKeys);
print(result); // prints ["Green", "Grey", "Light Green"]
-- UPDATE --
To implement this in Python, PyMongo's API supports all of the features of MongoDB’s map/reduce engine thus you could try the following:
import pymongo
import re
from bson.code import Code
client = pymongo.MongoClient("localhost", 27017)
db = client.test
map = Code("function () {"
" if (!this.colours) return;"
" this.colours.forEach(function (c){"
" emit(c.colourname.trim(), 1)"
" });"
"};")
reduce = Code("function (key, values) {"
" var count = 0;"
" for (index in values) {"
" count += values[index];"
" }"
" return count;"
" };")
result = db.myCollection.map_reduce(map, reduce, "map_reduce_result")
regx = re.compile("Gre", re.IGNORECASE)
for doc in result.find({"_id": regx}):
print(doc)

how to query sub document in mongodb?

how to query the sub document in sub document.?
how to retreive the name of the city in commercial document.
{
"_id" : " c2 ",
"commercial" : {
" type " : " restaurant ",
" sale_type " : " sale ",
" owner_name " : " josi schmit",
" address " : {
" street " : " kleine rittergasse ",
" plot_no " : 4,
" city " : " frankfurt ",
" state " : " hessen ",
" country " : "germany ",
"postal_code " : 60329,
"email" : " josi123#gmail.com"
},
" total_area " : " 300 sq meters ",
" sale_price(EUR) " : 100000,
" features " : {
" lifts " : 1,
" heating " : true,
" parking " : true,
" kitchen " : true,
" security_cameras " : true,
" furniture " : true
}
}
}
here is my query but i knew i'm wrong please help me.
db.property.find( {"_id": "c2" , "address.city": "c2"})
Regards
Sreekanth
Actualy there are two methods to do that:
db.property.find ( { "_id": "c2", "commercial.address.city": "c2" }, {"commercial.address.city": true} )
db.property.find ( { "_id": "c2", "commercial": { "address" : { "city": "c2" }}} )
The difference in both cases is that in first case mongo will follow the index if you have an index commercial.address.city, the second one will not, so I sugest you use the first one