I have a question regarding best practices to insert Documents in MongoDb.
In my data source the key "myData2" can be null or a string. Should I add "myData2" as null to my database or is it better to leave the value out if not defined? What is the "clean" way to deal with this?
[{
"myData1": "Stuff",
"myData2": null
}]
Since MongoDB permits fields to be added to documents at any time, most (production) applications are written to handle both of the following cases:
A new field is added to the code, but the existing data doesn't have it, and it needs to be added over time to the existing data either on demand or as a background process
A field is no longer used by the code but still contains values in the database
What would your application do if the field is missing, as opposed to if it's set to the null value? If it would do the same thing, then I suggest not setting fields to null values for two reasons:
It streamlines the code because you only need to handle one possibility (missing field) on the reading side, instead of two (field missing or null)
It requires less storage space in the database.
Related
I have an app which allows for dynamic table and column generation. One feature we are looking to implement is change tracking for each row. Anytime any changes occur, we want to keep track of the field and what the new value is.
One potential solution we are looking at is adding a "history" JSONB column to the table which would contain an ongoing array of every change. When a field is updated or multiple fields are updated, we could append a new element to the history field by using the || append syntax which will append the element to the existing jsonb vlaue:
update table set history = history || '{new node with changes}'
When viewing the "history changes", it would all be contained in this JSONB field and we'd have logic to parse out /display the changes over time.
Note: We won't need to query this JSONB column. It's simply the place to store all the row level changes. It seems like this should be an efficient way of saving updates.
My question is, is this a viable solution in regards to performance over time? It's possible this history field could become large over time, so would making any row changes become slower the larger the field? Or does the fact we are using the || operator to append data to the field mitigate any performance issues?
FYI, I am currently on Postgres 11, but could upgrade to newer versions if that had an impact.
Thanks for any feedback you can provide.
Trying to sync up a postgres record to airtable on create/update. The field has a couple of ids that I would like to check for in airtable to determine whether I should create a new record or update an existing one. The first id (optional_id) I need to search on can possibly be null. This causes the search to fail before it can get to the other id(required_id) that should always be populated. Is there any way I can skip the initial search for optional_id if it turns out to be null in postgres?
My current outline is as follows:
I would use a Formatter > Text > Default Value step in case the input value can be null and then make sure the fallback value is from a record that does not exist.
If further help is needed, feel free to reach out to us here:
https://zapier.com/app/get-help
I'm using Hasura as backend for react app.
In my SQL table I set up a field, let's say "identification_number" as [text, unique, nullable].
So identification_number might be null, but if it is not null, it should be unique across the table.
But sometimes fronted sends me data with an empty string as a value of my field.
{
name: "asdfomw",
year: "2010",
identification_number: "",
description: "some description"
}
In this situation Hasura successfully adds record to database.
But if an empty string is present in the payload for some other record, I get an error of "unique violation", because there are two records with equal identification_number having values of "".
So I think of two possible solutions for this problem:
Automatic conversion of empty-strings to nulls.
Adding constraints to Hasura scheme, not allowing empty strings for certain columns.
And I don't have answers for any of this questions.
You can add constraints like that in permissions if you'd like. That might be a bit too hacky of a solution.
The real question here is why are you allowing the frontend to upload values with an empty string if the backend does not accept that? Fail early. In your front end you can do the conversion or throw an error.
If you must do the conversion server side, you can write an event trigger to listen for insert/updates to this table and convert those values as they are inserted. I imagine you might still have issues if you try to insert several in one transaction with the same empty string value. You can also run into race conditions but I don't know for certain...
I am trying to save a null value in a dateTimeField object in MongoDB. I have a date I only want set after a certain condition has been fulfilled, but my site has the ability to save an object multiple time to the database before this variable will be set to anything other than a null or empty string value. I tried using db.dateTimeField(null=True) in my model file per Mongo's documentation to save a null value, but I still get an error when trying to save to the database.
According to this issue on Github, there is some inconsistent behavior with setting null values to dateTimeFields. Does anyone know if this has been fixed? I have a work around (adding my attribute to my object, then assigning it a value right before saving, therefore bypassing the need to save a null value, but it is a bit hacky, and I would like to use MongoDB's built in functionality if possible).
Thanks in advance!
I wonder, How do I change a live data schema with MongoDB ?
For example If I have "Users" collection with the following document:
var user = {
_id:123312,
name:"name",
age:12,
address:{
country:"",
city:"",
location:""
}
};
now, in a new version of my application, if I add a new property to "User" entity, let us say weight, tall or adult ( based on users year ), How to change all the current live data which does not have adult property. I read MapReduce and group aggregation command but, they seem to be comfortable and suitable for analytic operation or other calculations, or I am wrong.
So what is the best way to change your current running data schema in MongoDB ?
It really depends upon your programming language. MongoDB is really good at having a dynamic schema. I think your pattern of thought at the moment is too SQL related whereby you believe that all rows, even if they do not yet have a value, must have the new field.
The reality is quite different. The rows which have nothing meaningful to put into them do not require the field and you can, in your application, just check to see if the returned document has a value, if not then you can assume, as in a fixed SQL schema, that the value is null.
So this is one aspect where MongoDB shines, is the fact that you don't have to apply that new field to the entire schema on demand, instead you can lazy fill it as data is entered by the user.
So just code the field into your application and let the user do the work for you.
The best way to add this field is to write a loop, in maybe the console close or on the primary of your replica (if you have one, otherwise just on the server), like so:
db.users.find().forEach(function(doc){
doc.weight = '44 stone';
db.users.save(doc);
});
That is currently the best way to do something like what your asking.