How to handle if document structure after production changes.
Suppose I had 500 documents like this:
{
name: ‘n1’
height: ‘h1’
}
Later if I decide to add all the documents in below format:
{
name: ‘n501’
height: ‘h501’
weight: ‘w501’
}
I am using cursor.All(&userDetails) to decode(deserialize) in Go to get the output of the query in struct userDetails. If I modify the structure of further documents and userDetails accordingly, it will fail for the first 500 documents?
How to handle this change?
If you add a new field to your struct, querying old documents will not fail. Since the old documents do not have the new field saved in MongoDB, querying them will give you struct values where the new field will be its zero value. E.g. if its type is string, it will be the empty string "", if it's an int field, it will be 0.
If it bothers you that the old documents do not have this new field, you may extend them in the mongo console like this:
db.mycoll.updateMany({ "weight": {$exists:false} }, { $set: {"weight": ""} } )
This command adds a new weight field to old documents where this field did not exist, setting them to the empty string.
Related
I'm reading the documentation about Timeseries in Mongodb v5 - v6 and I don't understand if it's possible to upsert a record after it has been saved; for example if I have a record like this (the "name" field is the "metadata" ):
{
_id: ObjectId("6560a0ef02a1877734a9df66")
timestamp: 2022-11-24T01:00:00.000Z,
name: 'sensor1',
pressure: 5,
temperature: 25
}
is it possible to update the value of the "pressure" field after the record has been saved?
From the official mongo documentation, inside the "Time Series Collection Limitations" section, I read that: The update command may only modify the metaField field value.
Is there a way to upsert also other field? Thanks a lot.
No, updating the pressure field in your example is impossible with update alone, and upsert doesn't exist for time series collections.
The only functions currently available for time series collections are Delete and Update, but they only work on the metaField values, so in your example, we can only update/rename 'sensor1'.
The only workaround I know to update values is as follows:
Get a copy of all documents matched on the metaField values.
Update desired values on the copied documents.
Delete the original documents from the database
Insert your new copy of the documents into the database.
Here's a way to update values on a time series collections, using the MongoDB Shell (mongosh)
First, we create a test database. The important part here is the metaField named "metadata." This field will be an object/dictionary that stores multiple fields.
db.createCollection(
"test_coll",
{
timeseries: {
timeField: "timestamp",
metaField: "metadata",
granularity: "hours"
}
}
)
Then we add some test data to the collection. Note the 'metadata' is an object/dictionary that stores two fields named
sensorName and sensorLocation.
db.test_coll.insertMany( [
{
"metadata": { "sensorName": "sensor1", "sensorLocation": "outside"},
"timestamp": ISODate("2022-11-24T01:00:00.000Z"),
"pressure": 5,
"temperature": 32
},
{
"metadata": { "sensorName": "sensor1", "sensorLocation": "outside" },
"timestamp": ISODate("2022-11-24T02:00:00.000Z"),
"pressure": 6,
"temperature": 35
},
{
"metadata": { "sensorName": "sensor2", "sensorLocation": "inside" },
"timestamp": ISODate("2022-11-24T01:00:00.000Z"),
"pressure": 7,
"temperature": 72
},
] )
In your example we want to update the 'pressure' field which currently holds the pressure value of 5. So, we need to find all documents where the metaField 'metadata.sensorName' has a value of 'sensor1' and store all the found documents in a variable called old_docs.
var old_docs = db.test_coll.find({ "metadata.sensorName": "sensor1" })
Next, we loop through the documents (old_docs), updating them as needed. We add the documents (updated or not) to a variable named updated_docs. In this example, we are looping through all 'sensor1' documents, and if the timestamp is equal to '2022-11-24T01:00:00.000Z' we update the 'pressure' field with the value 555 ( which was initially 5 ). Alternatively, we could search for a specific _id here instead of a particular timestamp.
Note that there is a 'pressure' value of 7 at the
timestamp 2022-11-24T01:00:00.000Z, as well, but its value will remain the same because we are only looping through all 'sensor1' documents, so the document with sensorName set to sensor2 will not be updated.
var updated_docs = [];
while (old_docs.hasNext()) {
var doc = old_docs.next();
if (doc.timestamp.getTime() == ISODate("2022-11-24T01:00:00.000Z").getTime()){
print(doc.pressure)
doc.pressure = 555
}
updated_docs.push(doc)
}
We now have a copy of all the documents for 'sensor1' and we have updated our desired fields.
Next, we delete all documents with the metaField 'metadata.sensorName' equal to 'sensor1' ( on an actual database, please don't forget to backup first )
db.test_coll.deleteMany({ "metadata.sensorName": "sensor1" })
And finally, we insert our updated documents into the database.
db.test_coll.insertMany(updated_docs)
This workaround will update values, but it will not upsert them.
I am working on a mongo statement for adding and updating values into an object within my document.
Here is my current statement. field and value changes depending what is getting passed in:
db.collection.update(id, {
$set: {
analysis : {[field]: value}
}
});
Here is an example of what a document could look like(there are potentially 20+ fields in analysis)
{
_id
analysis:{
interest_rate: 22
sales_cost: 4000
value: 300
}
}
The problem is that every time I update the object all fields are removed except the the field I updated.
so if
field = interest_rate
and the new
value = 33
my document would end up looking like this and all the other fields in analysis would be removed:
{
_id
analysis:{
interest_rate: 33
}
}
Is there a way to update fields within an object like this to keep the code simple or will I have to write out update statements for each individual field?
You should use the dot notation to build the path when you're trying to update nested field. Try:
let fieldPath = 'analysis.' + field; // for instance "analysis.interest_rate"
db.collection.update(id, {
$set: {
fieldPath: value
}
});
Otherwise you're just replacing existing analysis object.
Although I think it is a general question, I could not find a solution that matches my needs.
I have 2 Mongo collections. The 'users' collection and the second one 'dbInfos'.
Now, I have a template called 'Infos' and want the already existing fields in the Mongo collections to be presented to the user in input fields in case there is data in the collection. When no data is provided in the database yet, it should be empty.
So here is my code, which works fine until I want to capture the fields from the second collection.
Template.Infos.onRendered(function() {
$('#txtName').val(Meteor.user().profile.name);
$('#txtEmail').val(Meteor.user().emails[0].address);
});
These 2 work great.
But I don´t know how to query the infos from the collection 'dbInfos', which is not the 'users' collection. Obviously Meteor.user().country does not work, because it is not in the 'users' collection. Maybe a find({}) query? However, I don´t know how to write it.
$('#txtCountry').val( ***query function***);
Regarding the structure of 'dbInfos': Every object has an _id which is equal to the userId plus more fields like country, city etc...
{
"_id": "12345",
"country": "countryX",
"city": "cityY"
}
Additionally, how can I guarantee that nothing is presented, when the field in the collection is empty? Or is this automatic, because it will just return an empty field?
Edit
I now tried this:
dbInfos.find({},{'country': 1, '_id': 0})
I think this is the correct syntax to retrieve the country field and suppress the output of the _id field. But I only get [object Object] as a return.
you're missing the idea of a foreign key. each item in a collection needs a unique key, assigned by mongo (usually). so the key of your country info being the same as the userId is not correct, but you're close. instead, you can reference the userId like this:
{
"_id": "abc123",
"userId": "12345",
"country": "countryX",
"city": "cityY"
}
here, "abc123" is unique to that collection and assigned by mongo, and "12345" is the _id of some record in Meteor.users.
so you can find it like this (this would be on the client, and you would have already subscribed to DBInfos collection):
let userId = Meteor.userId();
let matchingInfos = DBInfos.find({userId: userId});
the first userId is the name of the field in the collection, the second is the local variable that came from the logged in user.
update:
ok, i think i see where you're getting tripped it. there's a difference between find() and findOne().
find() returns a cursor, and that might be where you're getting your [object object]. findOne() returns an actual object.
for both, the first argument is a filter, and the second argument is an options field. e.g.
let cursor = DBInfos.find({
userId: Meteor.userId()
},
{
fields: {
country: 1
}
});
this is going to:
find all records that belong to the logged in user
make only the country and _id fields available
make that data available in the form of a cursor
the cursor allows you to iterate over the results, but it is not a JSON object of your results. a cursor is handy if you want to use "{{#each}}" in the HTML, for example.
if you simply change the find() to a findOne():
let result = DBInfos.findOne({ /** and the rest **/
... now you actually have a JSON result object.
you can also do a combination of find/fetch, which works like a findOne():
let result = DBInfos.find({
userId: Meteor.userId()
},
{
fields: {
country: 1
}
}).fetch();
with that result, you can now get country:
let country = result.country;
btw, you don't need to use the options to get country. i've been assuming all this code is on the client (might be a bad assumption). so this will work to get the country as well:
let result = DBInfos.findOne({userId: Meteor.userId()});
let country = result.country;
what's going on here? it's just like above, but the result JSON might have more fields in it than just country and _id. (it depends on what was published).
i'll typically use the options field when doing a find() on the server, to limit what's being published to the client. on the client, if you just need to grab the country field, you don't really need to specify the options in that way.
in that options, you can also do things like sort the results. that can be handy on the client when you're going to iterate on a cursor and you want the results displayed in a certain order.
does all that make sense? is that what was tripping you up?
Is it good or bad practice to keep the standard "_id" generated my mongo in a document as well as my own unique identifier such as "name", or should I just replace _id generated with the actual name so my documents will look like this:
{
_id: 782yb238b2327b3,
name: "my_name"
}
or just like this:
{
_id: "my_name"
}
This depends on the scenario, there is nothing wrong with having your own unique ID, it may be string or a number, completely depends on your situation as long as its unique, the important thing is you are in charge of it. You would want to add an index to it of course.
for example i have an additional ID field which is a number called 'ID', because i required a sequential number as an identifier, another usecase may be that your migrating an application so you have to conform to a particular sequence pattern.
The sequences for the unique identifies could easily be stored in a separate document/collections.
There is no issue with using the built in _id if you have no requirement not to have a custom one, an interesting fact is that you can get the created date out of the _id. Always useful.
db.col.insert( { name: "test" } );
var doc = db.col.findOne( { name: "test" } );
var timestamp = doc._id.getTimestamp();
I don't know why but if i try to update an existing field using the $set method, any existing fields are replaced in the same context.
For example. Say i have an existing collection with the following fields.
Name of collection: Ticket
{profile: {name: "Test", placement: 1}, requestor: _id}
When i attempt to add/update fields to this collection like this:
var ticket = Meteor.tickets.findOne({_id: ticketID});
if(ticket){
Meteor.users.update(ticket, {
$set: profile: {name: "Test2", new_fields: "value"}
});
}
The collection gets updated and the name field changes but placement is removed and no longer there. This is also true if i remove the name field. How do we properly update a meteor collection without having to keep passing the same structure over and over?
Just do this:
$set: {"profile.name": "Test2", "profile.new_fields": "value"}
I.e. You were replacing the whole hash. Instead you can update the fields within the hash.
if the field you want to change have a unique index, you can modify that particular field to what you want without destroying the remaining information in the field.
db.artists.find()
{"_id":ObjectId("1"),"name":"A1","media_id":["m1","m2" ]}
{"_id":ObjectId("2"),"name":"A2","media_id":["m2","m3"]}
{"_id":ObjectId("3"),"name":"A3","media_id":["m3","m1","m2"]}
db.artists.ensureIndex({"name":1})
db.artists.update(
{name:"A1"},
{$set: { name:"A4"}},
{ upsert: true }
)
b.artists.find()
{"_id":ObjectId("1"),"name":"A4","media_id":["m1","m2" ]}
{"_id":ObjectId("2"),"name":"A2","media_id":["m2","m3"]}
{"_id":ObjectId("3"),"name":"A3","media_id":["m3","m1","m2"]}
I am myself quite new in MongoDB but this worked pretty well for me.