I am a newbie to Mongodb and Pentaho, I am facing trouble to convert my existing RDBMS table to mongodb.
The structure of RDBMS is:
user_id,question_id,option_id
12,23,4
12,24,7
12,24,8
12,25,9
I want this to be converted into:
{
user_id:12,
questions:[
{question_id:23,Options:[4]},
{question_id:24}, Options:[7,8]},
{question_id:25},Optioins:[9]
]
}
I am using pentaho mongodbOutput, I tried various combinations but none worked.
One of the combination I have used is below
user_id,,Y,N,Y,N/A,Insert&Update
question_id,Questions[0],Y,N,N,$push,Insert&Update
options_id,Questions[1],Y,N,N,$push,Insert&Update
with this above settings I have got
{
user_id:12,
questions:[
{question_id:23,Options:4},
{question_id:24}, Options:7},
{question_id:24}, Options:8},
{question_id:25},Options:9
]
}
But I need Options 7,8 to be array.
Related
I am using MongoDB Compass version 1.25.0.
I was trying to insert a document shared by one of my colleague using the "Insert Document" feature of the MongoDB Compass.
But, it's showing that the document is not in correct format.
I think it is validating JSON format and ObjectId() is not a valid JSON value.
I am aware that Compass would create ObjectId() automatically but I want to pass it explicitly.
I resorted to the Shell and I was able to insert the document.
My question is does MongoDB Compass allow entering ObjectId() from the UI?
If yes, what am I doing wrong?
I was able to insert a document with ObjectId using below syntax:
{
"_id": {
"$oid": "60261ccf416a1ed478d7357a"
}
}
I found that the documentation of a bit off in clarifying how the $oid is supposed to be used.
At least for someone beginner like me.
We are migrating the data from Oracle to Mongo DB using Talend tool and we would need to add the object Id to each object inside an array. We have tried to use attribute #type with fixed value as ObjectId but it didn't worked.
We need the output as below:
{
"_id":"12243",
"name": "ABCD",
"city":"XYZ",
"requests":[
{
"_id" : ObjectId("5efdcf15ea9355c419fc9699"), // How to generate this ObjectId using talend tool in Mongo
"type":"department",
"value":"Science"
},
{
"_id" : ObjectId("K279kkqasj8ac023878hjc"), // How to generate this ObjectId using talend tool in Mongo
"type":"department",
"value":"Commerce"
}
]
}
Based on your needs, I assume that manually generating the ObjectId should be enough. I will propose to use:
either the standard MongoDB BSON java library (recommended)
or to generate this element by yourself as soon as you follow the official MongoDB conventions: https://docs.mongodb.com/manual/reference/bson-types/#objectid
The recommended way means that you have to add the MongoDB BSON Java library to your Talend project, certainly by including it as a JAR (see links below). I will not explain how (-> out-of-scope). Then simply do the following to add a correct _id to your embedded elements:
ObjectId id = new ObjectId();
// or
ObjectId id = ObjectId.get();
Related materials:
https://docs.mongodb.com/manual/reference/method/ObjectId/
How to generate unique object id in mongodb
https://mongodb.github.io/mongo-java-driver/4.0/bson/installation-guide/
https://mvnrepository.com/artifact/org.mongodb/bson
I'm trying to retrieve below mongo collection data using TalendOpenStudio for Big Data 7.2.
T
{
"id": "5b69b66d3dae73000fa39440",
"data":"Testing"
"products": [{ "orderid":"1234"
}],
"createDate": "2018-08-07T15:10:37.570Z",
"updateDate": "2018-08-09T16:09:46.621Z"
}
I'm able to get Id,data but when i try to get products i'm unable to get the data. On top of it i'm getting products as below
[Document{{"orderid"="1234"}}] this is blocking me for parsing it as json. Can someone help. I think its a basic mistake but as i said i'm new to Talend OS for Big Data
If anyone has already parsed this , can you please share the schema to be defined for products arraylist in talend and how they parsed id,data,products.
I tried using extractjson fields, using mongoschema from repository still no luck.
Looks like its a mongo driver(3.5.x) issue. Here are more details.
https://community.talend.com/t5/Design-and-Development/TmongoDbInput-schema-not-retreiving-properly/m-p/202163#M110456
This question already has an answer here:
Why does mongoose use schema when mongodb's benefit is supposed to be that it's schema-less?
(1 answer)
Closed 5 years ago.
I am a beginner with MongoDB and trying to learn MEAN Stack. So I am using Mongoose as the ORM
I read that MongoDB is a NoSQL database, but while using Mongoose as ORM, I am asked to create a schema first. Why is it so? There shouldn't be a schema ideally as MongoDB is a NoSQL database.
Thanks in advance.
Mongoose is an orm on top of mongodb , if you are using core mongodb you need not create any schema , you can just dump any data you want , but in mongoose you have a schema so that you can i have some basic key value pair for advanced searching and filtering and you can anytime update the schema. Or If you want to go schemaless and dump whatever the response is you can use a schema type like this var someSchema = {data:Object} and drop all your data in this data key and then you can easily extract whatever JSON data is inside your id field.
var mongoose = require('mongoose');
module.exports = mongoose.model('twitter', {
created_at:{
type:Date
},
dump:{
type:Object
}
});
In the above example dump is used to save whatever JSON I get as a response from twitter api and created_at contains only the creating date of tweet , so I have the entire data , but if i want to search tweets of a particular date I can search it using a find query on created_at and this query will be lot faster and here I have a fixed structure and a knowledge about what to expect of a find query each time a run one, So this is one of the benefit of using the mongoose orm i.e I don't lose data but I can maximise my searching ability by creating appropriate keys.
So basically mongoose is an ORM db , it offers you relational db features like creating foreign keys , not strictly foreign keys but you can create something like an id reference to another schema and later populate the field by the id associated parameters when you fetch data using your find query , also a relational schema is easy to manage , what mongoose does is it gives a JSON/BSON based db the power of relational db and you get best of both the world i.e you can easily maintain new keys or you don't need to worry about extracting each and every data from your operation and placing it properly/inserting it , you just need to see that your keys and values match , as well as you have flexibility in update operations while having a schema or table structure.
I use mongo for saving my attributes , now the question is i want to save the attributes like this
{
title:"new product"
...
...
attr:[
{
name:"color",value:"red"
},
{
name:"size",value:6
},
....
]
}
now if i create index for value field is it bad index design?
should i separate the integer fields from string fields and then index separately ?
MongoDB encapsulates features of dynamic schema which supports variance of different data types across documents in MongoDB collection. Regarding concern mentioned above to store values with different data types in one field ,I suggest that it can be stored and its main purpose for which MongoDB is designed.