The most common thing I do in the mongo DB shell is find objects by ID, eg:
db.collection.find({_id: ObjectId("55a3e051dc75954f0f37c2f2"})
I do this over and over and I find having to wrap the id with ObjectId over and over gets old. I wish I had a findById-like shorthand form like what mongoose provides. I feel like the shell ought to be smart enough to figure out what I mean here for example:
db.collection.find("55a3e051dc75954f0f37c2f2")
How might I do this? Or are there any alternative ways of querying by id in the mongo shell?
Fortunately, you can extend the shell quite easily, for example by adding the following method to the ~/.mongorc.js file which is executed when you start the mongo client:
DBCollection.prototype.findById = function(id) {
return db.getCollection(this._shortName).find( { "_id" : ObjectId(id) } );
}
Then you can execute something like db.collection.findById("55a3e051dc75954f0f37c2f2")
The shorthand for find({_id: ObjectId("...")}) is find(ObjectId("...")).
Related
I am trying to do a simple find in Panache but I'm stuck with the wildcard operator.
I have:
Model.find("payload.tags.name = ?1", "tag-to")
.stream()
.map(m -> (Model) m)
.collect(Collectors.toList());
and my document looks something like this:
{
...
payload:Object{
swagger:"2.0"
info:Object
host:"petstore.swagger.io"
basePath:"/v2"
tags:Array[
0:Object [
name:"tag-to-find"
description:"a tag i want to find"
]
]
}
}
When I try to find "tag-to-find" it works, but I don't know how to get the wildcards going. In mongoshell i just use db.Model.find({"payload.tags.name": /ag-to-/}) and it works.
What you are using in Mongo shell is a JavaScript regex.
You should also be able to use it with MongoDB with Panache.
You should normaly use the regex with the $regex operator, not sure how Mongo shell handle it but the following should work:
Model.list("payload.tags.name like ?1", "/tag-to/")
I use .list() instead of find() as it directly return the list of documents.
The query used here is what we called PanacheQL query that will maps to a MongoDB native query, you can also use a native query directly (with named or indexed parameters).
Simplified query is explained here: https://quarkus.io/guides/mongodb-panache#simplified-queries
Since version 3.6 MongoDB requires the use of cursor or explain in aggregate queries. It's a breaking change so I have to modify my earlier code.
But when I add cursor or explain to my query, the request simply enters an endless loop and MongoDB never responds. It doesn't even seem to time out.
This simple aggregation just hangs the code and never responds:
db.collection('users').aggregate([{ $match: { username: 'admin' }}],
{ cursor: {} },
(err, docs) => {
console.log('Aggregation completed.');
});
I can replace { cursor: {} } with { explain: true } and the result is the same. It works perfectly under older MongoDB versions without this one parameter.
Without cursor or explain I get this error message:
The 'cursor' option is required, except for aggregate with the explain argument
I'm not the only one who ran into this:
https://github.com/nosqlclient/nosqlclient/issues/419
OK, this was a little tricky, but finally it works. Looks like there are some major breaking changes in MongoDB's Node.js driver which nobody bothered to tell me.
1. The Node.js MongoDB driver has to be upgraded. My current version is 3.0.7.
2. The way how MongoDB connects has been changed, breaking any old code. The client connection now returns a client object, not merely the db. It has to be handled differently. There is a SO answer explaining it perfectly:
db.collection is not a function when using MongoClient v3.0
3. Aggregations now return an AggregationCursor object, not an array of data. Instead of a callback now you have to iterate through it.
var cursor = collection.aggregate([ ... ],
{ cursor: { batchSize: 1 } });
cursor.each((err, docs) => {
...
});
So it seems you have to rewrite ALL your db operations after upgrading to MongoDB 3.6. Yay, thanks for all the extra work, MongoDB Team! Guess that's where I'm done with your product.
I want to call a custom python function on some existing attribute of every document in the entire collection and store the result as a new key-value pair in that (same) document. May I know if there's any way to do that (since each call is independent of others) ?
I noticed cursor.forEach but can't it be done just using python efficiently ?
A simple example would be to split the string in text and store the no. of words as a new attribute.
def split_count(text):
# some complex preprocessing...
return len(text.split())
# Need something like this...
db.collection.update_many({}, {'$set': {"split": split_count('$text') }}, upsert=True)
But it seems like setting a new attribute in a document based on the value of another attribute in the same document is not possible this way yet. This post is old but the issues seem to be still open.
I found a way to call any custom python function on a collection using parallel_scan in PyMongo.
def process_text(cursor):
for row in cursor.batch_size(200):
# Any complex preprocessing here...
split_text = row['text'].split()
db.collection.update_one({'_id': row['_id']},
{'$set': {'split_text': split_text,
'num_words': len(split_text) }},
upsert=True)
def preprocess(num_threads=4):
# Get up to max 'num_threads' cursors.
cursors = db.collection.parallel_scan(num_threads)
threads = [threading.Thread(target=process_text, args=(cursor,)) for cursor in cursors]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
This is not really faster than cursor.forEach (but not that slow either), but it helps me execute any arbitrarily complex python code and save the results from within Python itself.
Also if I have an array of ints in one of the attributes, doing cursor.forEach converts them to floats which I don't want. So I preferred this way.
But I would be glad to know if there're any better ways than this :)
It is quite unlikely that it will ever be efficient to do this kind of thing in python. This is because the document would have to make a round trip and go through the python function on the client machine.
In your example code, you are passing the result of a function to a mongodb update query, which won't work. You can't run any python code inside mongodb queries on the db server.
As the answer to you linked question suggests, this type of action has to be performed in the mongo shell. e.g:
db.collection.find().snapshot().forEach(
function (elem) {
splitLength = elem.text.split(" ").length
db.collection.update(
{
_id: elem._id
},
{
$set: {
split: splitLength
}
}
);
}
);
My application has the following stack:
Sinatra on Ruby -> MongoMapper -> MongoDB
The application puts several entries in the database. In order to crosslink to other pages, I've added some sort of syntax. e.g.:
Coffee is a black, caffeinated liquid made from beans. {Tea} is made from leaves. Both drinks are sometimes enjoyed with {milk}
In this example {Tea} will link to another DB entry about tea.
I'm trying to query my mongoDB about all 'linked terms'. Usually in ruby I would do something like this: /{([a-zA-Z0-9])+}/ where the () will return a matched string. In mongo however I get the whole record.
How can I get mongo to return me only the matched parts of the record I'm looking for. So for the example above it would return:
["Tea", "milk"]
I'm trying to avoid pulling the entire record into Ruby and processing them there
I don't know if I understand.
db.yourColl.aggregate([
{
$match:{"yourKey":{$regex:'[a-zA-Z0-9]', "$options" : "i"}}
},
{
$group:{
_id:null,
tot:{$push:"$yourKey"}
}
}])
If you don't want to have duplicate in totuse $addToSet
The way I solved this problem is using the string aggregation commands to extract the StartingIndexCP, ending indexCP and substrCP commands to extract the string I wanted. Since you could have multiple of these {} you need to have a projection to identify these CP indices in one shot and have another projection to extract the words you need. Hope this helps.
I have an app using Mongoid on top of MongoDB and an update is failing silently.
The code looks like this:
# Are we getting a new attribute? Yes we are!
p "attr first name = #{#attributes['first_name']}"
if user.update_attributes!(#attributes)
u = User.find(params[:id]).to_json
end
No exception is thrown in this code. So I looked at my MongoDB log and constructed this query based on what mongo is trying to do:
db.users.update({ "_id": "4d5561276ce886c496000001" }, { $set: { "first_name": "Erinamodobo" } }, false);
Now this does not cause any exceptions but when I grab the record that was supposed to be updated with this query:
db.users.find({"email":"escharling#somecompany.com"})
I see that the "first_name" attribute has not been updated.
Any idea why this could be happening? Sounds like something stupid.
Thanks!
You need to find out what
user.update_attributes!(#attributes)
is actually doing. This could be an issue with Mongoid. When you configure Mongoid, you can set up a logger. There you should be able to see what the driver is writing to MongoDB, and that should help answer your question. The next best thing is to post your code to the Mongoid mailing list (http://groups.google.com/group/mongoid) where people who work with Mongoid all the time will probably know what's going on.
Updating Mongoid to the latest rc.7 fixed this issue
Enable "safe mode" on your operations.