I can query mongodb directly using this query {key: {$in: [/faq/]}} but how do I replicate this when using Filters#in?
in("key", Seq("/faq/")) doesn't work as it gets passed as a string.
Related
I am trying to use MongoDB aggregation in a Meteor application. I am using Meteor 1.11, which uses MongoDB 4.2.8. I built and tested my aggregation pipeline with Robo 3T and also tested it with the Mongo shell. It works fine in those contexts, but I can't get it working in a Meteor method.
I narrowed the problem down to the match stage, which is first in the pipeline. When running in a Meteor method, this stage appears to match all documents in the collection. When run in Robo 3T, it returns 4 documents. Here's how I run it in a Meteor method:
MyCollection
.rawCollection()
.aggregate([{$match: {vocabularyName: "IPF", type: "T"}}])
.toArray()
.then(docs => {
console.log(`got ${docs.length} docs`);
console.log(docs[0]);
console.log(docs[1]);
})
.catch(error => console.log(`error ${error} in db_methods`));
Every document in MyCollection contains a "type" field with varying values; most (but not all) contain a "vocabularyName" field. When I run this in a Meteor method, the returned array size matches the total document count and the first two results documents contain no vocabularyName field at all, and "type: 'S'". Which does not make any sense.
On a guess, I also tried this pipeline, but it made no difference:
[ {"$match": {"$and": [{ "vocabularyName": "IPF"}, {"type": "T"}]}} ]
Newby MongoDB & CosmosDB user here, I've read the answer to this question How does MongoDB treat find().sort() queries with respect to single and compound indexes? and the offocial MongoDB docs and I believe my index creation mirrors that answer so I am leaning towards this being a CosmosDB issue but reading their documentation CosmosDB 3.6 supports compounded indexes as well, so I am at a loss right now.
I am able to run sort() queries like db.Videos.find().sort({"PublishedOn": 1}) from the mongo command line on a collection with an index created as db.Videos.createIndex({"PublishedOn": 1}) or db.Videos.createIndex({"PublishedOn": -1}).
And when I add a 'where' clause to the find like this db.Videos.find({"IsPinned": false}).sort({"PublishedOn": 1}) the above index still works.
However I now have document look ups which I want to avoid, so I drop the above single field index and create a compounded index like this db.Videos.createIndex({"IsPinned": 1, "PublishedOn": 1}) or db.Videos.createIndex({"PublishedOn": 1, "IsPinned": 1}) but now the query always fails with the error The index path corresponding to the specified order-by item is excluded..
Is this a limitation of CosmosDB or is my 'ordering' in the index bad?
The issue with CosmosDB is that it expects all WHERE fields to be used in the GROUP BY clause as well in exactly the same order else it won't use the index.
Creating an index as db.Videos.createIndex({"IsPinned": 1, "PublishedOn": 1}) and then updating the query to be db.Videos.find({"IsPinned": false}).sort({"IsPinned": 1, "PublishedOn": 1}) works like a charm.
I inferred this from reading the CosmosDB documentation on indexing policies (https://learn.microsoft.com/en-us/azure/cosmos-db/index-policy) as the MongoDB documentation suddenly stops after the index creation (https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-indexing) section.
I am trying to query a binary field in mongo db. The data looks like this:
{"_id":"WE8fSixi8EuWnUiThhZdlw=="}
I've tried a lot of things for example:
{ '_id': new Binary( 'WE8fSixi8EuWnUiThhZdlw==', Binary.SUBTYPE_DEFAULT) }
{ '_id': Binary( 'WE8fSixi8EuWnUiThhZdlw==', 0) }
etc
Nothing seems to be working, have exhausted google and the mongo documentation, any helper would be amazing.
UPDATE:
Now you should be able to query UUID and BinData from MongoDB Compass v1.20+ (COMPASS-1083). For example: {"field": BinData(0, "valid_base64")}.
PREVIOUS:
I see that you're using MongoDB Compass to query the field. Unfortunately, the current version of MongoDB Compass (v1.16.x) does not support querying binary data.
You can utilise mongo shell to query the data instead. For example:
db.collection.find({'_id':BinData(0, "WE8fSixi8EuWnUiThhZdlw==")});
Please note that the field name _id is reserved for use as a primary key; its value must be unique in the collection, and is immutable. Depending on the value of the binary that you're storing into _id, I would suggest to store the binary in another field and keep the value of _id to contain ObjectId.
I have a query that works in MongoDB Shell but it is throwing an "aggregation result exceeds maximum document size (16MB)" exception using Spring Data MongoDB when I increase the number of documents to return.
Here is the shell query that returns without error:
db.myCollection.aggregate([{"$skip":0},{"$limit":10000},{"$match":{"_id"="550b2552e4b03562d6329a39"}}], {"allowDiskUse":true})
Here is the Spring Data snippet:
List<AggregrationOperation> ops = new ArrayList<AggregationOperation>();
ops.add(new SkipOperation(0));
ops.add(new LimitOperation(10000));
ops.add(new MatchOperation(Criteria.where("_id").is("550b2552e4b03562d6329a39")));
TypeAggregation<MyCollection> aggregation = Aggregation.newAggregation(MyCollection.class, ops).withOptions(Aggregation.newAggregationOptions().allowDiskUse(true).build());
AggregationResults<MyCollection> result = mongoTemplate.aggregate(aggregation, MyCollection.class);
List<MyCollection> myResults = result.getMappedResults()
When I limit to say 100 or 1000 records everything works fine. When I increase to 10,000 I get the "aggregation result exceeds maximum document size (16MB)" exception.
For reference here are the versions I am using:
MongoDB = 3.0.0
Mongo Java Driver = 2.13.0
Spring Data MongoDB = 1.6.0
Spring Data Commons = 1.9.0
UPDATE:
Background: I came to the aggregate solution because I was exceeding the 32MB sort limit when using a find(). I understand that adding an index will solve this problem. Unfortunately this solution doesn't scale. I want to have sorting on all of our columns in our List Grids which would mean 10+ columns to index. Of course from the UI I could limit sorting to specific columns but once again I am trying to avoid that solution and hence the reason I tried an aggregation.
It appears that using a cursor is my only solution. Can anyone confirm that Spring Data MongoDB does not provide direct cursor support meaning I would have to use MongoDB's API?
I had the same issue using RoboMongo (mongo client).
Try using db.runCommand() instead of .aggregate().
Is seems that using the aggregate method on some clients will add some data to the query and allowDiskUse will be ignored.
When I used ranCommand, it worked fine:
db.runCommand(
{ aggregate: "collection_name",
pipeline: [
{$group: ...},
{$match: ...},
{$group: ...}
],
allowDiskUse: true
}
You can use outputMode(AggregationOptions.OutputMode.CURSOR) as ouput option instead of the limited 16MB document by default. See Aggregation Cursor Usage.
Aggregation.newAggregationOptions().outputMode(AggregationOptions.OutputMode.CURSOR).allowDiskUse(true).build()
The mongodb $or operator works as intended outside of a meteorjs context:
db.users.find({$or: [{email: 'some#mail.com'},{city: 'atlanta'}]});
I get results for any document that has email some#mail.com or city of atlanta.
The same query in Meteor syntax doesn't yield the same results :
Users = new Meteor.Collection("users");
Users.find({$or: [{email: 'some#mail.com'},{city: 'atlanta'}]});
I've read the meteor docs - http://docs.meteor.com/#find - and since it doesn't say anything about it, I'm assuming it should run just the same as a mongodb 1.6+ instance?
find returns a cursor object. You need to use a fetch to get the array of values. Try:
console.log(Users.find({$or: [{email: 'some#mail.com'},{city: 'atlanta'}]}).fetch());