Given a List of mongo document UUIDs, I would like to validate if these IDs have corresponding documents in the DB. What is the efficient way to perform bulk validation?
Lets say my program has 100 id strings and would like to validate all these IDs in one DB call. Is it feasible?
I could do it sequentially (i.e using exists(T id) methods in spring data repository) but I would rather want to do in one call.
My program uses spring data for mongodb. But I am open to any native mongo command. I can code the Query in spring data for it. Also, If any of the ID in the list does not have a document in DB, I want to get such ID in response.
I don't know spring-data but you can use the $in operator and do something like find({ _id : { $in : [ your, ids, here ] } }) and then work from there. If you need a simple yes/no for all of them existing, you can just check the length of your result set vs. the length of your input array.
Related
I'm fairly new to mongodb but I was wondering if there's a way by which we can get 2 different results from same mongodb collection in one database call uisng mongo java driver with morphia.
I have a collection accounts and I'm fetching data based on a key accountId. I need below two results/outputs from this collection in one query.
count of all the documents where accountID is 'xyz'
ResultList of first N documents where accountID is 'xyz' AND resultSet is sorted by a timestamp field.
to resolve the second scenario I'm using:
..Query....limit(N).order("TimeField").field("TimeField").filter("accountID =", "xyz").asList();
This is working fine as per expectation but to get the total count (scenario 1) of all documents with accountId = 'xyz' needs another mongodb call, which I want to avoid.
MongoDB doesn't support such batching on queries, unfortunately. You'll have to execute two separate calls.
Say I have objects in a MongoDB collection:
{
...
"json" : "{\"things\":[2494090781803658355,5114030115038563045,3035856943768375362,8931213615561493991,7574631742057150605,480863244020297489]}"
}
It's an Azure "MongoDB" so doesn't support all the features, but suppose it does.
This search will find that document:
db.coll.find({"json" : {$regex : "5114030115038563045|8931213615561493991"}})
Of course, it's scanning the whole collection to pull these records out. What's an efficient/faster way to find documents where the list of "things"
contains any of a list of "things" in a query? It seems like throwing a search engine like Solr or ElasticSearch would solve this, and perhaps
using another Azure's Data Lake storage would make this more searchable, so I'm considering those options. They're outside the scope of this
question though; I'd like to know if there's a Mongo-ish way to search this collection by index.
The only option you have available to you if you're storing a JSON string is to use a text index with a $text operator.
If this document structure isn't set in stone, however, you might consider also separately storing the JSON as a nested subdocument (with the appropriate sanitation, of course). This would allow you to construct an index on json.things, while still storing the JSON string, and allow you to perform a query on e.g. "json.things": {$in: [ "5114030115038563045", "8931213615561493991" ]}
I'm using cloudant, with no auth, Cors enabled.
it works very well, Limit and skip working good.
but i can't find how to search for something .
I'm trying to find a document where cp is 24000 , for example with this query :
https://1c54473b-be6e-42d6-b914-d0ecae937981-bluemix.cloudant.com/etablissements/_all_docs?skip=0&limit=10&include_docs=true&q=cp:24000
But, the query doesn't return the right document.
I've also tried
https://1c54473b-be6e-42d6-b914-d0ecae937981-bluemix.cloudant.com/etablissements/_all_docs?skip=0&limit=10&include_docs=true&_search({'cp':24000})
with no luck.
oh, and by the way, do you know if jquery.couch.js lib has been discontinued? I cant even find it on github, nor on my hard disk while im using foxant, and it is not in the directory also..
The /db/_all_docs endpoint hits the primary index of the database where all of the documents in the database can be found in _id order.
If you wish to query the database to get a subset of the data you have three options
Cloudant Query - hit the POST /db/_find endpoint passing in a JavaScript object containing the selector which defines the query you wish to perform (like the WHERE clause of a SQL query) e.g. {selector: {cp: 24000}}
MapReduce - create a Map function in a design document that filters the documents you are interested it. It creates a materialized view that can be queried and filtered later. e.g. function(doc){ emit(doc.cp, null);}
Cloudant Search - this uses the Apache Lucene library to generate an index on the fields you specify. You can then query the index: q=cp:24000, which looks similar to the query you are looking to perform.
In my MongoDB 3.2 based application I want to perform the documents processing. In order to avoid the repeated processing on the same document I want to update its flag and update this document in the database.
The possible approach is:
Query the data: FindIterable<Document> documents = db.collection.find(query);.
Perform some business logic on these documents.
Iterate over the documents, update each document and store it in a new collection.
Push the new collection to the database with db.collection.updateMany();.
Theoretically, this approach should work but I'm not sure that it is the optimal scenario.
Is there any way in MongoDB Java API to perform the followings two operations:
to query documents (to get them from the DB and to pass to the separate method);
to update them and then store the updated version in DB;
in a more elegant way comparing to the proposed above approach?
You can update document inplace using update:
db.collection.update(
{query},
{update},
{multi:true}
);
It will iterate over all documents in the collection which match the query and updated fields specified in the update.
EDIT:
To apply some business logic to individual documents you can iterate over matching documents as following:
db.collection.find({query}).forEach(
function (doc) {
// your logic business
if (doc.question == "Great Question of Life") {
doc.answer = 42;
}
db.collection.save(doc);
}
)
If for example I keep lists of user posts in redis, for example a user has 1000 posts, and the posts documents are stored into mongodb but the link between the user and the posts is stored inside redis, I can rtetrieve the array containing all the ids of a user post from redis, but what is the efficient way to retrieving them from mongodb?
do I pass a parameter to mongoDB with the array of ids, and mongo will fetch those for me?
I don't seem to find any documentation on this, if Anyone is willing to help me out!
thanks in advance!
To retrieve a number of documents per id, you can use the $in operator to build the MongoDB query. See the following section from the documentation:
http://docs.mongodb.org/manual/reference/operator/query/in/#op._S_in
For instance you can build a query such as:
db.mycollection.find( { _id : { $in: [ id1, id2, id3, .... ] } } )
Depending on how much ids will be returned by Redis, you may have to group them in batch of n items (n=100 for instance) to run several MongoDB queries. IMO, this is a bad practice to build such query containing more than a few thousands ids. It is better to have smaller queries but accept to pay for the extra roundtrips.