I am faced with a problem where I am not sure which path to take. So I am asking it here. I have an application where there can be products and there can be metadata for a product. These metadata can be created and deleted from the front end. So let's say, today a each product has two meta data (e.g. Name, Price) then tomorrow it can be three, four or more or even less than two. So this is dynamic. I am trying to represent the data as follows
Product =
{
"_id": mongo
"Name": string
"Description": string
"BasePrice": number
"CreatedBy": user mongo _id
"CreatedAt": timestamp
"ModifiedAt": timestamp
"MetaData": BSON object (having all the keys from ProductMetadata collection and their values. e.g. {"Category": "table ware", "Material": "oak wood, copper", "Length": 5.6})
}
ProductMetadata =
{
"_id": mongo
"Name": string (e.g. - "Category" or "Material" or "Height")
"Type": string (indicating what kind of value it can hold like string/integer/array. e.g. - "string")
"CreatedAt": timestamp
"ModifiedAt": timestamp
}
As you can see this is a pure dynamic situation so having a structure at the code level which represent the model is impossible.
My problem is, how do I implement such a thing using mgo and Go lang? If I need to use reflection then can some one please point me towards a good blog/tutorial where I can get a little bit more info. Or if you think that there is a fundamental problem in the approach of modeling the data then please correct me so that it can be easily implemented using Go.
In Python, it would not be a problem to implement this. I know that. But I am confused about the Go implementation.
Please help.
Thanks in advance
If Keys for metadata are unique, why not just use a map.
meaning your Product struct looks like:
struct Product {
ID bson.ObjectId `bson:"_id,omitempty"`
Name string
Description string
... omitted other fields ...
MetaData map[string]map[string]interface{} // map of string -> map of string -> anything
}
If you can have more than one instance of a given meta data, ie: 2 categories, use a list:
struct Product {
ID bson.ObjectId `bson:"_id,omitempty"`
Name string
Description string
... omitted other fields ...
MetaData []map[string]interface{} // list of maps of string -> anything
}
Related
I have the following model:
type UserModel struct {
Id string `bson:"_id,omitempty"`
CreatedAt *time.Time `bson:"createdAt,omitempty"`
BasicInfo *UserBasicInfoModel `bson:"basicInfo,omitempty"`
}
// *Embedded document*
type UserBasicInfoModel struct {
FirstName *string `bson:"firstName,omitempty"`
LastName *string `bson:"lastName,omitempty"`
}
I am using pointers, in order to be able to distinguish between a missing value (nil) and default value (eg empty strings, false values etc). I also use omitempty to be able to do partial updates.
When I create a user I get the following (correct) response back:
"id": "aba19b45-5e84-55e0-84f8-90fad41712f6",
"createdAt": "2018-05-26T15:08:56.764453386+03:00",
"basicInfo": {
"firstName": "Initial first name",
"lastName": "Initial last name"
}
When I try to update the document though I have an issue.
I send the changes as a new UserModel, to only change the FirstName field in the embedded document like this:
newFirstName := "New Value"
UserModel{
BasicInfo: &UserBasicInfoModel{
FirstName: &newFirstName,
},
}
The code I use to do the update is the following:
UpdateId(id, bson.M{"$set": changes})
The response I get back is the following:
"id": "aba19b45-5e84-55e0-84f8-90fad41712f6",
"createdAt": "2018-05-26T12:08:56.764Z",
"basicInfo": {
"firstName": "New Value",
"lastName": null
}
The createdAt value is not null (as I expected) however the lastName value is null (which is not what I expected)
I would have expected to get back the following:
"id": "aba19b45-5e84-55e0-84f8-90fad41712f6",
"createdAt": "2018-05-26T12:08:56.764Z",
"basicInfo": {
"firstName": "New Value",
"lastName": "Initial last name"
}
What can I do to achieve a partial update in a subdocument using mgo?
First let's quickly explain your createdAt field. This is the value you save: 2018-05-26T15:08:56.764453386+03:00. Know that MongoDB stores dates with millisecond precision, and in UTC timezone. So this date when saved and retrieved from MongoDB becomes 2018-05-26T12:08:56.764Z, this is the "same" time instant, just in UTC zone and precision truncated to milliseconds.
Now on to updating embedded documents:
The short and unfortunate answer is that we can't do this directly with the mgo library and Go models.
Why?
When we use the ,omitempty option, and we leave some pointer fields at their zero value (that is, being nil), it's like if we were using a value whose type didn't even have those fields.
So in your example, if you only change the BasicInfo.FirstName field, and you use this value to update, it is equivalent to using these structures:
type UserModel struct {
Id string `bson:"_id,omitempty"`
BasicInfo *UserBasicInfoModel `bson:"basicInfo,omitempty"`
}
type UserBasicInfoModel struct {
FirstName *string `bson:"firstName,omitempty"`
}
So the effect of your issued update command will be the following:
db.users.update({_id: "aba19b45-5e84-55e0-84f8-90fad41712f6"},
{$set:{
"_id": "aba19b45-5e84-55e0-84f8-90fad41712f6",
"basicInfo": {
"firstName": "New Value"
}
}}
)
What does this mean? To set the _id to the same value (it won't change), and to set the basicInfo field to an embedded document which only has a single firstName property. This will erase the lastName field of the embedded basicInfo document. So when you unmarshal the document after the update into a value of your UserModel type, the LastName field will remain nil (because it is not present in MongoDB anymore).
What can we do?
Flatten the embedded document
One trivial solution is to not use an embedded document, but add fields of UserBasicInfoModel to UserModel:
type UserModel struct {
Id string `bson:"_id,omitempty"`
CreatedAt *time.Time `bson:"createdAt,omitempty"`
FirstName *string `bson:"firstName,omitempty"`
LastName *string `bson:"lastName,omitempty"`
}
Hybrid with ,inline option
This solution keeps the separate Go struct, but in MongoDB it will not be an embedded document (BasicInfo will be flattened just like in the previous example):
type UserModel struct {
Id string `bson:"_id,omitempty"`
CreatedAt *time.Time `bson:"createdAt,omitempty"`
BasicInfo UserBasicInfoModel `bson:"basicInfo,omitempty,inline"`
}
Note that BasicInfo needs to be a non-pointer if ,inline is used. This is not a problem, as we can leave it being an empty struct if its fields are not to be changed, since its fields are pointers, so leaving them nil will not change them.
Doing "manual" update
If you do need to use embedded document, the mgo library allows you to update specific fields of embedded documents, but then you have to "manually" construct the update document, like in this example:
c.UpdateId(Id, bson.M{"$set": bson.M{
"basicInfo.firstName": newFirstName,
}})
Yes, this isn't convenient at all. If you do need this many times with different types, you may create a utility function that uses reflection, iterates over the fields recursively, and assemble the update document from fields which are not nil. Then you could pass that dynamically generated update doc to UpdateId() for example.
I'd like to fetch snapshot which contains typed text. For example node look like this
"Team": {
"Yankees": {
"uid1": "name",
"uid2": "name"
},
"Angels": {
"uid1": "name"
"uid3": "name"
}
and if user typed yan in search bar then I want to fetch "Yankees" snapshot. I saw some document and stack over flow post and tried like so
ref.child("Team").queryStarting(atValue: "yan").queryEnding(atValue: "yan\u{f8ff}").observe
but it doesn't work. How can I do this? Thank you!
Firebase searches are case sensitive. Since your key starts with an uppercase Y, the query only matches if it also does that:
ref.child("Team")
.queryOrderedByKey()
.queryStarting(atValue: "Yan")
.queryEnding(atValue: "Yan\u{f8ff}").observe
I also queryOrderedByKey() to be explicit about what you want to order/filter on.
If you want to allow case-insensitive filtering, the typical approach is to add a property with the all-lowercase value to each team:
"Team": {
"Yankees": {
"teamNameForSearch": "yankees",
"uid1": "name",
"uid2": "name"
},
"Angels": {
"teamNameForSearch": "angels",
"uid1": "name"
"uid3": "name"
}
Now you can search with:
ref.child("Team")
.queryOrdered(byChild: "teamNameForSearch")
.queryStarting(atValue: "yan")
.queryEnding(atValue: "yan\u{f8ff}").observe
A final note is that both approaches only do so-called prefix matches: they find teams whose name starts with what the user typed. If you want a contains operation (as the title of your question suggests), you will have to look beyond Firebase for a solution. For more on that, see Kato's answer here: Firebase query - Find item with child that contains string
You need to change the db to this:
"Team": {
"randomid": {
"team":"Yankees",
"uid1": "name",
"uid2": "name"
},
"randomid": {
"team":"Angels"
"uid1": "name"
"uid3": "name"
}
and now you can do this:
ref.child("Team").queryOrdered(byChild: "team").queryStarting(atValue: "Yan").queryEnding(atValue: "Yan\u{f8ff}").observe
First in your query above, you need to use queryOrdered() to know which child you want to order.
Also in your database the node Team is not equal to anything, it is a parent node that has child nodes.
So to fix this, the node Team needs to be equal to a value(like in the database in this answer) so you will be able to order it and use queryStarting and queryEnding on it.
Although I think it is a general question, I could not find a solution that matches my needs.
I have 2 Mongo collections. The 'users' collection and the second one 'dbInfos'.
Now, I have a template called 'Infos' and want the already existing fields in the Mongo collections to be presented to the user in input fields in case there is data in the collection. When no data is provided in the database yet, it should be empty.
So here is my code, which works fine until I want to capture the fields from the second collection.
Template.Infos.onRendered(function() {
$('#txtName').val(Meteor.user().profile.name);
$('#txtEmail').val(Meteor.user().emails[0].address);
});
These 2 work great.
But I don´t know how to query the infos from the collection 'dbInfos', which is not the 'users' collection. Obviously Meteor.user().country does not work, because it is not in the 'users' collection. Maybe a find({}) query? However, I don´t know how to write it.
$('#txtCountry').val( ***query function***);
Regarding the structure of 'dbInfos': Every object has an _id which is equal to the userId plus more fields like country, city etc...
{
"_id": "12345",
"country": "countryX",
"city": "cityY"
}
Additionally, how can I guarantee that nothing is presented, when the field in the collection is empty? Or is this automatic, because it will just return an empty field?
Edit
I now tried this:
dbInfos.find({},{'country': 1, '_id': 0})
I think this is the correct syntax to retrieve the country field and suppress the output of the _id field. But I only get [object Object] as a return.
you're missing the idea of a foreign key. each item in a collection needs a unique key, assigned by mongo (usually). so the key of your country info being the same as the userId is not correct, but you're close. instead, you can reference the userId like this:
{
"_id": "abc123",
"userId": "12345",
"country": "countryX",
"city": "cityY"
}
here, "abc123" is unique to that collection and assigned by mongo, and "12345" is the _id of some record in Meteor.users.
so you can find it like this (this would be on the client, and you would have already subscribed to DBInfos collection):
let userId = Meteor.userId();
let matchingInfos = DBInfos.find({userId: userId});
the first userId is the name of the field in the collection, the second is the local variable that came from the logged in user.
update:
ok, i think i see where you're getting tripped it. there's a difference between find() and findOne().
find() returns a cursor, and that might be where you're getting your [object object]. findOne() returns an actual object.
for both, the first argument is a filter, and the second argument is an options field. e.g.
let cursor = DBInfos.find({
userId: Meteor.userId()
},
{
fields: {
country: 1
}
});
this is going to:
find all records that belong to the logged in user
make only the country and _id fields available
make that data available in the form of a cursor
the cursor allows you to iterate over the results, but it is not a JSON object of your results. a cursor is handy if you want to use "{{#each}}" in the HTML, for example.
if you simply change the find() to a findOne():
let result = DBInfos.findOne({ /** and the rest **/
... now you actually have a JSON result object.
you can also do a combination of find/fetch, which works like a findOne():
let result = DBInfos.find({
userId: Meteor.userId()
},
{
fields: {
country: 1
}
}).fetch();
with that result, you can now get country:
let country = result.country;
btw, you don't need to use the options to get country. i've been assuming all this code is on the client (might be a bad assumption). so this will work to get the country as well:
let result = DBInfos.findOne({userId: Meteor.userId()});
let country = result.country;
what's going on here? it's just like above, but the result JSON might have more fields in it than just country and _id. (it depends on what was published).
i'll typically use the options field when doing a find() on the server, to limit what's being published to the client. on the client, if you just need to grab the country field, you don't really need to specify the options in that way.
in that options, you can also do things like sort the results. that can be handy on the client when you're going to iterate on a cursor and you want the results displayed in a certain order.
does all that make sense? is that what was tripping you up?
I'm very new to MongoDB, i tell you sorry for this question but i have a problem to understand how to create a document that can contain a value with different "type:
My document can contain data like this:
// Example ONE
{
"customer" : "aCustomer",
"type": "TYPE_ONE",
"value": "Value here"
}
// Example TWO
{
"customer": "aCustomer",
"type": "TYPE_TWO",
"value": {
"parameter1": "value for parameter one",
"parameter2": "value for parameter two"
}
}
// Example THREE
{
"customer": "aCustomer",
"type": "TYPE_THREE",
"value": {
"anotherParameter": "another value",
{
"someParameter": "value for some parameter",
...
}
}
}
Customer field will be even present, the type can be different (TYPE_ONE, TYPE_TWO and so on), based on the TYPE the value can be a string, an object, an array etc.
Looking this example, i should create three kind of collections (one for type) or the same collection (for example, a collection named "measurements") can contain differend kind of value on the field "value" ?
Trying some insert in my DB instance i dont get any error (i'm able to insert object, string and array on property value), but i would like to know if is the correct way...
I come from RDBMS, i'm a bit confused right now.. thanks a lot for your support.
You can find the answer here https://docs.mongodb.com/drivers/use-cases/product-catalog
MongoDB's dynamic schema means that each need not conform to the same schema.
Lets take a simple "bad" example : lets assume I have 2 collections 'person' and 'address'. And lets assume in 'address' I want to store '_id' of the person the address is associated with. Is there any benefit to store this "referential key" item as ObjectId vs string in 'address' collection?
I feel like storing them as string should not hurt but I have not worked in mongo for very long and do not know if it will hurt down the road if I follow this pattern.
I read the post here : Store _Id as object or string in MongoDB?
And its said that ObjectId is faster, and I assume its true if you are fetching/updating using the ObjectId in parent collection(for eg. fetching/updating 'person' collection using person._id as ObjectId), but I couldn't find anything that suggests that same could be true if searching by string id representation in other collection(in our example search in address collection by person._id as string)
Your feedback is much appreciated.
Regardless of performance, you should store the "referential key" in the same format as the _id field that you are referring too. That means that if your referred document is:
{ _id: ObjectID("68746287..."), value: 'foo' }
then you'd refer to it as:
{ _id: ObjectID(…parent document id…), subDoc: ObjectID("68746287...")
If the document that you're pointing to has a string as an ID, then it'd look like:
{ _id: "derick-address-1", value: 'foo' }
then you'd refer to it as:
{ _id: ObjectID(…parent document id…), subDoc: "derick-address-1" }
Besides that, because you're talking about persons and addresses, it might make more sense to not have them in two documents altogether, but instead embed the document:
{ _id: ObjectID(…parent document id…),
'name' : 'Derick',
'addresses' : [
{ 'type' : 'Home', 'street' : 'Victoria Road' },
{ 'type' : 'Work', 'street' : 'King William Street' },
]
}
As for use string as id of document, in meteor collection, you could generate the document id either Random.id() as string or Meteor.Collection.ObjectID() as ObjectId.
In this discussion loop, Mongodb string id vs ObjectId, here is one good summary,
ObjectId Pros
it has an embedded timestamp in it.
it's the default Mongo _id type; ubiquitous
interoperability with other apps and drivers
ObjectId Cons
it's an object, and a little more difficult to manipulate in practice.
there will be times when you forget to wrap your string in new ObjectId()
it requires server side object creation to maintain _id uniqueness
- which makes generating them client-side by minimongo problematic
String Pros
developers can create domain specific _id topologies
String Cons
developer has to ensure uniqueness of _ids
findAndModify() and getNextSequence() queries may be invalidated
All those information above is based on the meteor framework. For Mongodb, it is better to use ObjectId, reasons are in the question linked in your question.
Storing it as objectId is benificial. It is faster as ObjectId size is 12 bytes compared to string which takes 24 bytes.
Also, You should try to de-normalize your collections so that you don't need to make 2 collections (Opposite to RDBMS).
Something like this might be better in general:
{ _id : "1",
person : {
Name : "abc",
age: 20
},
address : {
street : "1st main",
city: "Bangalore",
country: "India"
}
}
But again, it depends on your use case. This might be not suitable sometimes.
Hope that helps! :)