Query to trim the data from existing mongodb database field - mongodb

I have a field DateOfBirth of type String. Currently i am storing the date in the format yyyy-MM-ddTHH:mm:ss Z.
Now i want to change the date format in the existing database as yyyy-MM-dd i don't want time now i just want to trim that how can i do that using a query in mongodb.
db.profile.update( {}, { $set : { "physical.dateOfBirth" : "2012-01-11" } }, true, true);
I tried using this query i did not get desired result.

Currently you cannot update values based on a current value in mongodd with the exception of increment and decrement.
You will first need to find each value and then update them before saving them again.
You could do it multiple ways:
get all profiles (only need _id and properties to update)
for each profile
set the value to its yyyy-MM-dd value
save profile
or
get all profiles (only need _id and properties to update)
create bulkOp
for each profile
set the value to its yyyy-MM-dd value
add update to bulkOp
execute bulkOp
or
aggregate to find distinct yyyy-MM-dd values
for each <yyyy-MM-dd value>
var match = {"physical.dateOfBirth": {$regex: <yyyy-MM-dd value>}};
var update = {$set: {"physical.dateOfBirth": <yyyy-MM-dd value>}};
var multi = {multi: true};
db.profiles.update(match, update, multi);
Clearly this is just the processes you could try and not actual code.
The last way would be the easiest on memory because you would not actually load much into memory, just the distinct values incase you have 100s or 1000s of values with the same dates.
Good luck
EDIT: something along the lines of this:
The $cond in the _id is just to match everything together in 1 item
var aggregate = db.profiles.aggregate({$group: {
_id: {$cond: ['_id', 'a', 'a']},
values: {$addToSet: { $dateToString: { format: '%Y-%m-%d', date: '$physical.dateOfBirth' } }}
}});
var values = aggregate[0].values;
for(var i = 0; i < values.length; i++) {
var date = new Date(values[i]);
var nextDay = new Date().setDate(date.getDate() + 1);
var match = {'physical.dateOfBirth': {$gte: date, $lte: nextDay}};
var update = {$set: {'physical.dateOfBirth': date}};
var multi = {multi: true};
db.profiles.update(match, update, multi);
}

Related

How to get the particuler object fields using ReativeMongo without a case class [duplicate]

In my MongoDB, I have a student collection with 10 records having fields name and roll. One record of this collection is:
{
"_id" : ObjectId("53d9feff55d6b4dd1171dd9e"),
"name" : "Swati",
"roll" : "80",
}
I want to retrieve the field roll only for all 10 records in the collection as we would do in traditional database by using:
SELECT roll FROM student
I went through many blogs but all are resulting in a query which must have WHERE clause in it, for example:
db.students.find({ "roll": { $gt: 70 })
The query is equivalent to:
SELECT * FROM student WHERE roll > 70
My requirement is to find a single key only without any condition. So, what is the query operation for that.
From the MongoDB docs:
A projection can explicitly include several fields. In the following operation, find() method returns all documents that match the query. In the result set, only the item and qty fields and, by default, the _id field return in the matching documents.
db.inventory.find( { type: 'food' }, { item: 1, qty: 1 } )
In this example from the folks at Mongo, the returned documents will contain only the fields of item, qty, and _id.
Thus, you should be able to issue a statement such as:
db.students.find({}, {roll:1, _id:0})
The above statement will select all documents in the students collection, and the returned document will return only the roll field (and exclude the _id).
If we don't mention _id:0 the fields returned will be roll and _id. The '_id' field is always displayed by default. So we need to explicitly mention _id:0 along with roll.
get all data from table
db.student.find({})
SELECT * FROM student
get all data from table without _id
db.student.find({}, {_id:0})
SELECT name, roll FROM student
get all data from one field with _id
db.student.find({}, {roll:1})
SELECT id, roll FROM student
get all data from one field without _id
db.student.find({}, {roll:1, _id:0})
SELECT roll FROM student
find specified data using where clause
db.student.find({roll: 80})
SELECT * FROM students WHERE roll = '80'
find a data using where clause and greater than condition
db.student.find({ "roll": { $gt: 70 }}) // $gt is greater than
SELECT * FROM student WHERE roll > '70'
find a data using where clause and greater than or equal to condition
db.student.find({ "roll": { $gte: 70 }}) // $gte is greater than or equal
SELECT * FROM student WHERE roll >= '70'
find a data using where clause and less than or equal to condition
db.student.find({ "roll": { $lte: 70 }}) // $lte is less than or equal
SELECT * FROM student WHERE roll <= '70'
find a data using where clause and less than to condition
db.student.find({ "roll": { $lt: 70 }}) // $lt is less than
SELECT * FROM student WHERE roll < '70'
I think mattingly890 has the correct answer , here is another example along with the pattern/commmand
db.collection.find( {}, {your_key:1, _id:0})
> db.mycollection.find().pretty();
{
"_id": ObjectId("54ffca63cea5644e7cda8e1a"),
"host": "google",
"ip": "1.1.192.1"
}
db.mycollection.find({},{ "_id": 0, "host": 1 }).pretty();
Here you go , 3 ways of doing , Shortest to boring :
db.student.find({}, 'roll _id'); // <--- Just multiple fields name space separated
// OR
db.student.find({}).select('roll _id'); // <--- Just multiple fields name space separated
// OR
db.student.find({}, {'roll' : 1 , '_id' : 1 ); // <---- Old lengthy boring way
To remove specific field use - operator :
db.student.find({}).select('roll -_id') // <--- Will remove id from result
While gowtham's answer is complete, it is worth noting that those commands may differ from on API to another (for those not using mongo's shell).
Please refer to documentation link for detailed info.
Nodejs, for instance, have a method called `projection that you would append to your find function in order to project.
Following the same example set, commands like the following can be used with Node:
db.student.find({}).project({roll:1})
SELECT _id, roll FROM student
Or
db.student.find({}).project({roll:1, _id: 0})
SELECT roll FROM student
and so on.
Again for nodejs users, do not forget (what you should already be familiar with if you used this API before) to use toArray in order to append your .then command.
Try the following query:
db.student.find({}, {roll: 1, _id: 0});
And if you are using console you can add pretty() for making it easy to read.
db.student.find({}, {roll: 1, _id: 0}).pretty();
Hope this helps!!
Just for educational purposes you could also do it with any of the following ways:
1.
var query = {"roll": {$gt: 70};
var cursor = db.student.find(query);
cursor.project({"roll":1, "_id":0});
2.
var query = {"roll": {$gt: 70};
var projection = {"roll":1, "_id":0};
var cursor = db.student.find(query,projection);
`
db.<collection>.find({}, {field1: <value>, field2: <value> ...})
In your example, you can do something like:
db.students.find({}, {"roll":true, "_id":false})
Projection
The projection parameter determines which fields are returned in the
matching documents. The projection parameter takes a document of the
following form:
{ field1: <value>, field2: <value> ... }
The <value> can be any of the following:
1 or true to include the field in the return documents.
0 or false to exclude the field.
NOTE
For the _id field, you do not have to explicitly specify _id: 1 to
return the _id field. The find() method always returns the _id field
unless you specify _id: 0 to suppress the field.
READ MORE
For better understanding I have written similar MySQL query.
Selecting specific fields
MongoDB : db.collection_name.find({},{name:true,email:true,phone:true});
MySQL : SELECT name,email,phone FROM table_name;
Selecting specific fields with where clause
MongoDB : db.collection_name.find({email:'you#email.com'},{name:true,email:true,phone:true});
MySQL : SELECT name,email,phone FROM table_name WHERE email = 'you#email.com';
This works for me,
db.student.find({},{"roll":1})
no condition in where clause i.e., inside first curly braces.
inside next curly braces: list of projection field names to be needed in the result and 1 indicates particular field is the part of the query result
getting name of the student
student-details = db.students.find({{ "roll": {$gt: 70} },{"name": 1, "_id": False})
getting name & roll of the student
student-details = db.students.find({{ "roll": {$gt: 70}},{"name": 1,"roll":1,"_id": False})
I just want to add to the answers that if you want to display a field that is nested in another object, you can use the following syntax
db.collection.find( {}, {{'object.key': true}})
Here key is present inside the object named object
{ "_id" : ObjectId("5d2ef0702385"), "object" : { "key" : "value" } }
var collection = db.collection('appuser');
collection.aggregate(
{ $project : { firstName : 1, lastName : 1 } },function(err, res){
res.toArray(function(err, realRes){
console.log("response roo==>",realRes);
});
});
it's working
Use the Query like this in the shell:
1. Use database_name
e.g: use database_name
2. Which returns only assets particular field information when matched , _id:0 specifies not to display ID in the result
db.collection_name.find( { "Search_Field": "value" },
{ "Field_to_display": 1,_id:0 } )
If u want to retrieve the field "roll" only for all 10 records in the collections.
Then try this.
In MongoDb :
db.students.find( { } , { " roll " : { " $roll " })
In Sql :
select roll from students
The query for MongoDB here fees is collection and description is a field.
db.getCollection('fees').find({},{description:1,_id:0})
Apart from what people have already mentioned I am just introducing indexes to the mix.
So imagine a large collection, with let's say over 1 million documents and you have to run a query like this.
The WiredTiger Internal cache will have to keep all that data in the cache if you have to run this query on it, if not that data will be fed into the WT Internal Cache either from FS Cache or Disk before the retrieval from DB is done (in batches if being called for from a driver connected to database & given that 1 million documents are not returned in 1 go, cursor comes into play)
Covered query can be an alternative. Copying the text from docs directly.
When an index covers a query, MongoDB can both match the query conditions and return the results using only the index keys; i.e. MongoDB does not need to examine documents from the collection to return the results.
When an index covers a query, the explain result has an IXSCAN stage that is not a descendant of a FETCH stage, and in the executionStats, the totalDocsExamined is 0.
Query : db.getCollection('qaa').find({roll_no : {$gte : 0}},{_id : 0, roll_no : 1})
Index : db.getCollection('qaa').createIndex({roll_no : 1})
If the index here is in WT Internal Cache then it would be a straight forward process to get the values. An index has impact on the write performance of the system thus this would make more sense if the reads are a plenty compared to the writes.
If you are using the MongoDB driver in NodeJs then the above-mentioned answers might not work for you. You will have to do something like this to get only selected properties as a response.
import { MongoClient } from "mongodb";
// Replace the uri string with your MongoDB deployment's connection string.
const uri = "<connection string uri>";
const client = new MongoClient(uri);
async function run() {
try {
await client.connect();
const database = client.db("sample_mflix");
const movies = database.collection("movies");
// Query for a movie that has the title 'The Room'
const query = { title: "The Room" };
const options = {
// sort matched documents in descending order by rating
sort: { "imdb.rating": -1 },
// Include only the `title` and `imdb` fields in the returned document
projection: { _id: 0, title: 1, imdb: 1 },
};
const movie = await movies.findOne(query, options);
/** since this method returns the matched document, not a cursor,
* print it directly
*/
console.log(movie);
} finally {
await client.close();
}
}
run().catch(console.dir);
This code is copied from the actual MongoDB doc you can check here.
https://docs.mongodb.com/drivers/node/current/usage-examples/findOne/
db.student.find({}, {"roll":1, "_id":0})
This is equivalent to -
Select roll from student
db.student.find({}, {"roll":1, "name":1, "_id":0})
This is equivalent to -
Select roll, name from student
In mongodb 3.4 we can use below logic, i am not sure about previous versions
select roll from student ==> db.student.find(!{}, {roll:1})
the above logic helps to define some columns (if they are less)
Using Studio 3T for MongoDB, if I use .find({}, { _id: 0, roll: true }) it still return an array of objects with an empty _id property.
Using JavaScript map helped me to only retrieve the desired roll property as an array of string:
var rolls = db.student
.find({ roll: { $gt: 70 } }) // query where role > 70
.map(x => x.roll); // return an array of role
Not sure this answers the question but I believe it's worth mentioning here.
There is one more way for selecting single field (and not multiple) using db.collection_name.distinct();
e.g.,db.student.distinct('roll',{});
Or, 2nd way: Using db.collection_name.find().forEach(); (multiple fields can be selected here by concatenation)
e.g., db.collection_name.find().forEach(function(c1){print(c1.roll);});
_id = "123321"; _user = await likes.find({liker_id: _id},{liked_id:"$liked_id"}); ;
let suppose you have liker_id and liked_id field in the document so by putting "$liked_id" it will return _id and liked_id only.
For Single Update :
db.collection_name.update({ field_name_1: ("value")}, { $set: { field_name_2 : "new_value" }});
For MultiUpdate :
db.collection_name.updateMany({ field_name_1: ("value")}, { $set: {field_name_2 : "new_value" }});
Make sure indexes are proper.

How to change type of a field in MongoDB

Here are my questions:
How to change all the string type "NULL" in one field to nulls in null type in MongoDB?
How to change all the string type "20xx-xx-xx" in one field to date type in MongoDB?
db.foo.insert({bar: "NULL", date: "2012-11-30"})
db.foo.insert({bar: "NULL", date: "2010-09-21"})
db.foo.insert({bar: "NOTNULL", date: "2010-09-21"})
How to change all the string type "NULL" in one field to nulls in null type in MongoDB?
db.foo.update({bar: "NULL"}, {$set: {bar: null}}, {multi: true})
How to change all the string type "20xx-xx-xx" in one field to date type in MongoDB?
db.foo.find({date: {$exists: true}}).forEach(
function(doc) {doc.date = new Date(doc.date); db.foo.save(doc)}
)
Edit
If for some reason you have problem with default parsing it can be done manually:
Lets assume you have document like this
doc = {date: "2012-01-30"}
First split string representation on hypens and cast each to integer
dateArray = doc.date.split('-').map(function(s) { return parseInt(s); })
Substract 1 from month field (months in js are counted from 0)
dateArray[1] -= 1
Now we can create new date object. There are whoever some things to consider. When you parse string to date, there is assumption that the string represents UTC date when you use this:
new Date("2012-01-30")
You will get
ISODate("2012-01-30T00:00:00Z")
You can also create Date constructor like this
new Date(dateArray[0], dateArray[1], dateArray[2])
but it will assume that data is using local tz so and in my case (GMT+1) I get:
ISODate("2012-01-29T23:00:00Z")
If it is not what you want you can set some fields manually:
date = new Date(0) // ISODate("1970-01-01T00:00:00Z")
date.setUTCFullYear(dateArray[0], dateArray[1], dateArray[2])
Now date looks like this:
ISODate("2012-01-30T00:00:00Z")
Wrapping it all together:
db.foo.find({date: {$exists: true}}).forEach(
function(doc) {
dateArray = doc.date.split('-').map(function(s) { return parseInt(s); });
dateArray[1] -= 1;
date = new Date(0);
date.setUTCFullYear(dateArray[0], dateArray[1], dateArray[2]);
doc.date = date;
db.foo.save(doc)
}
)

Aggregate MongoDB results by ObjectId date

How can I aggregate my MongoDB results by ObjectId date. Example:
Default cursor results:
cursor = [
{'_id': ObjectId('5220b974a61ad0000746c0d0'),'content': 'Foo'},
{'_id': ObjectId('521f541d4ce02a000752763a'),'content': 'Bar'},
{'_id': ObjectId('521ef350d24a9b00077090a5'),'content': 'Baz'},
]
Projected results:
projected_cursor = [
{'2013-09-08':
{'_id': ObjectId('5220b974a61ad0000746c0d0'),'content': 'Foo'},
{'_id': ObjectId('521f541d4ce02a000752763a'),'content': 'Bar'}
},
{'2013-09-07':
{'_id': ObjectId('521ef350d24a9b00077090a5'),'content': 'Baz'}
}
]
This is what I'm currently using in PyMongo to achieve these results, but it's messy and I'd like to see how I can do it using MongoDB's aggregation framework (or even MapReduce):
cursor = db.find({}, limit=10).sort("_id", pymongo.DESCENDING)
messages = [x for x in cursor]
this_date = lambda x: x['_id'].generation_time.date()
dates = set([this_date(message) for message in messages])
dates_dict = {date: [m for m in messages if this_date(m) == date] for date in dates}
And yes, I know that the easiest way would be to simply add a new date field to each record then aggregate by that, but that's not what I want to do right now.
Thanks!
Update: There is a built in way to do this now, see https://stackoverflow.com/a/51766657/295687
There is no way to accomplish what you're asking with mongodb's
aggregation framework, because there is no aggregation operator that
can turn ObjectId's into something date-like (there is a JIRA
ticket, though). You
should be able to accomplish what you want using map-reduce, however:
// map function
function domap() {
// turn ObjectId --> ISODate
var date = this._id.getTimestamp();
// format the date however you want
var year = date.getFullYear();
var month = date.getMonth();
var day = date.getDate();
// yields date string as key, entire document as value
emit(year+"-"+month+"-"+day, this);
}
// reduce function
function doreduce(datestring, docs) {
return {"date":datestring, "docs":docs};
}
The Jira Ticket pointed out by llovett has been solved, so now you can use date operators like $isoWeek and $year to extract this information from an ObjectId.
Your aggregation would look something like this:
{
"$project":
{
"_id": {
"$dateFromParts" : {
"year": { "$year": "$_id"},
"month": { "$month": "$_id"},
"day": { "$dayOfMonth": "$_id"}
}
}
}
}
So this doesn't answer my question directly, but I did find a better way to replace all that lambda nonsense above using Python's setdefault:
d = {}
for message in messages:
key = message['_id'].generation_time.date()
d.setdefault(key,[]).append(message)
Thanks to #raymondh for the hint in is PyCon talk:
Transforming Code into Beautiful, Idiomatic Python

mongoose compund unique index does not work

I have following index:
PaymentSchema.index({ driver_id: 1, year: 1, month: 1 },{ unique: true });
So I want this collection to hold just one record for each different combination of fields driver_id, year and month. I want to update collection with upsert option:
var query = {
driver_id: req.params.driver_id,
year: req.params.year,
month: req.params.month,
amount: req.params.old_value
};
var update = {
$set: {
amount: req.params.new_value
}
};
var options = {
upsert: true
};
Payment.update(query,update,options,function(err,rows){
if(err) return next(err);
res.json({});
});
So what I want is to update document with given unique key (driver_id+year+month) and with additional condition amount = .... If query conditions are ok document should be updated - and it works. If document is not found according to this conditions and document with unique index does not exist it is created. But if document with unique index exists (only amount condition is incorrect) then a new document in created with same unique index (driver_id + year + month). It is strange because I declared unique index on those 3 fields (driver_id+year+month) and I can see in mongoshell that there exist two documents with those fields the same...
Solved: I had to restart mongod and delete database (probably reindex option would work too).

Mongo add timestamp field from existing date field

I currently have a collection with documents like the following:
{ foo: 'bar', timeCreated: ISODate("2012-06-28T06:51:48.374Z") }
I would now like to add a timestampCreated key to the documents in this collection, to make querying by time easier.
I was able to add the new column with an update and $set operation, and set the timestamp value but I appears to be setting the current timestamp using this:
db.reports.update({}, {
$set : {
timestampCreated : new Timestamp(new Date('$.timeCreated'), 0)
}
}, false, true);
I however have not been able to figure out a way to add this column and set it's value to the timestamp of the existing 'timeCreated' field.
Do a find for all the documents, limiting to just the id and timeCreated fields. Then loop over that and generate the timestampCreated value, and do an update on each.
Use updateMany() which can accept aggregate pipelines (starting from MongoDB 4.2) and thus take advantage of the $toLong operator which converts a Date into the number of milliseconds since the epoch.
Also use the $type query in the update filter to limit only documents with the timeCreated field and of Date type:
db.reports.updateMany(
{ 'timeCreated': {
'$exists': true,
'$type': 9
} },
[
{ '$set': {
'timestampCreated': { '$toLong': '$timeCreated' }
} }
]
)