I am trying to get some random items from the database using the Sample() method. I updated to the latest version of the driver and copied the using statements from the linked example. However, something isn't working, and I am hoping it's some simple mistake on my part. All the relevant information is in the image below:
Edit:
Greg, I read the aggregation docs here and the raw db method doc here, but I still don't get it. Last two lines are my attempts, I have never used aggregation before:
var mongoCollection = GetMongoCollection<BsonDocument>(collectionName);
long[] locationIds = new long[] { 1, 2, 3, 4, 5 };
var locationsArray = new BsonArray();
foreach (long l in locationIds) { locationsArray.Add(l); };
FilterDefinition<BsonDocument> sampleFilter = Builders<BsonDocument>.Filter.In("LocationId", locationsArray);
var findSomething = mongoCollection.Find(sampleFilter); //this works, the two attempts below don't.
var aggregateSomething = mongoCollection.Aggregate(sampleFilter).Sample(25);
var aggregateSomething2 = mongoCollection.Aggregate().Match(sampleFilter).Sample(25);
Sample is only available from an aggregation. You need to start with Aggregate, not Find. I believe it's also available in Linq.
UPDATE:
Looks like we don't have a method for it specifically. However, you can use the AppendStage method.
mongoCollection.Aggregate(sampleFilter)
.AppendStage<BsonDocument>("{ $sample: { size: 3 } }");
Related
The following code is giving the above error, and I cannot figure out why:
var x = _sqlConn.Connection().QueryResults<Results>("MyDb.dbo.get_records", new { id = theId });
int retVal = x.Outputs.Return_Value;
if (retVal == 0) // ...meaning result set was also returned...fine to this point.
{
var list = x.Outputs.Set1; // exception thrown here with above error
var temp = list.FirstOrDefault();
I have been using other features of Insight.Database for a number of years, but have not had to retrieve a SQL RETURN value at the same time as a recordset. The SQL itself works correctly in SSMS, returning a result set and the RETURN value of 0, as expected. This is happening in VS2019, .NET 4 and .NET 4.5.2; Insight.Database 5.2.7 and 5.2.8.
I got this code from the following page:
https://github.com/jonwagner/Insight.Database/wiki/Specifying-Result-Structures
where it shows this:
var results = connection.QueryResults<Beer, Glass>("GetAllBeersAndAllGlasses");
IList<Beer> beers = results.Set1;
which I combined with the following code from here:
https://github.com/jonwagner/Insight.Database/wiki/Output-Parameters
var results = connection.QueryResults<Results>("MyProc", inputParameters);
var p = results.Outputs.p;
That part works. It's accessing .Set1 that is failing, and I am not sure how to track down why.
I do not have experience with the FastExpando class, but Jon promised magic, and I want to believe. Thanks for any help.
I haven’t tried results+dynamic objects in a while…
I think it is because you are doing:
QueryResults<Results> and Results is an Insight type
You probably want:
QueryResults<MyType>
And then you get back a Results<MyType>
Which contains the return val and Set1
If not, post a ticket over on github and we will help you out.
I'm starting to learn about mongoose/MongoDB aggregation functions, and am having some basic difficulties. For example, I'm trying to do the following:
var myModels= require('./models/myModel');
var myCount = myModels.countDocuments({userID: "A"});
console.log(myCount );
I just want to count the number of documents with userID of "A" but when this prints to the console, it's printing as a whole object, instead of just a numerical count. I've read the answer here but I'm still not able to solve this problem (also, is there a way, unlike in that question, to return the count directly rather than having to predefine a variable and set it in a callback function?)
I'm trying to follow the guide here and don't see where I'm going wrong.
It's because the return value of countDocuments is a promise and not a number.
You either need to wait for that Promise or use callback syntax like so:
var myModels= require('./models/myModel');
// this required the code to be inside an async function
var myCount = await myModels.countDocuments({userID: "A"});
console.log(myCount);
Or:
var myModels= require('./models/myModel');
myModels.countDocuments({userID: "A"})
.then((myCount) =>{console.log(myCount);});
How to have inline results from a mapReducet with the mongodb java driver 3.2?
with driver version 2.x I was doing:
DBColleciont coll = client.getDB(dbName).getCollection(collName);
coll.mapReduce(map, reduce, null, OutputType.INLINE, query);
the new 3.x driver has two mapReduce() methods returning MapReduceIterable which misses a method to specify the INLINE output mode.
MongoCollection<Documetn> coll = client.getDatabase(dbName).getCollection(collName)
coll
.mapReduce(map, reduce).
.filter(query);
You can create the map-reduce command manually:
String mapFunction = ...
String reduceFunction = ...
BsonDocument command = new BsonDocument();
BsonJavaScript map = new BsonJavaScript(mapFunction);
BsonJavaScript red = new BsonJavaScript(reduceFunction);
BsonDocument query = new BsonDocument("someidentifier", new BsonString("somevalue"));
command.append("mapreduce", new BsonString("mySourceCollection"));
command.append("query", query);
command.append("map", map);
command.append("reduce", red);
command.append("out", new BsonDocument("inline", new BsonBoolean(true)));
Document result = mongoClient.getDatabase(database).runCommand(command);
I think this is extremely ugly, but it is the only working solution I found so far using 3.2. (... and would be very interested in a better variant, too... ;-))
I think I found it...
I had a deeper look into mongodb's Java driver source and it seems that the INLINE output feature is implicitly accessible:
The class MapReduceIterableImpl<TDocument, TResult>(MapReduceIterableImpl.java), which is the default implementation of the Interface return type of mapReduce(),
holds a private boolean inline with initial value true.
The only place where this can ever be switched to false is the method collectionName(final String collectionName) for which the description is as follows:
Sets the collectionName for the output of the MapReduce
The default action is replace the collection if it exists, to change this use action(com.mongodb.client.model.MapReduceAction).
If you never call this method on the object instance after mapReduce(), it will remain true as initialized...meaning: if there is no output collection, it must be inline.
Later on, when you access your result with iterator(), first(), forEach(...) etc internally the execute() method gets called which hast the magic if condition:
if (inline) {
MapReduceWithInlineResultsOperation<TResult> operation =
new MapReduceWithInlineResultsOperation<TResult>(namespace,
new BsonJavaScript(mapFunction),
new BsonJavaScript(reduceFunction),
codecRegistry.get(resultClass))
.filter(toBsonDocument(filter))
.limit(limit)
.maxTime(maxTimeMS, MILLISECONDS)
.jsMode(jsMode)
.scope(toBsonDocument(scope))
.sort(toBsonDocument(sort))
.verbose(verbose)
.readConcern(readConcern);
....
} else {
MapReduceToCollectionOperation operation =
new MapReduceToCollectionOperation(namespace, new BsonJavaScript(mapFunction), new BsonJavaScript(reduceFunction),
collectionName)
.filter(toBsonDocument(filter))
.limit(limit)
.maxTime(maxTimeMS, MILLISECONDS)
.jsMode(jsMode)
.scope(toBsonDocument(scope))
.sort(toBsonDocument(sort))
.verbose(verbose)
.action(action.getValue())
.nonAtomic(nonAtomic)
.sharded(sharded)
.databaseName(databaseName)
.bypassDocumentValidation(bypassDocumentValidation);
...so it is instanciating MapReduceWithInlineResultsOperation when collectionName() has not been called.
I had no chance to test it because my NetBeans hates me at the moment, but I think it is pretty clear.
What do you think, did I miss something?
Would be glad if I could help you shift the code to API 3.x, great project!
For some reason I am unable to use Select() after a Skip()/Take() unless I do this in a certain way. The following code works and allows me to use result as part of a sub query.
var query = QueryOver.Of<MyType>();
query.Skip(1);
var result = query.Select(myType => myType.Id);
However, if I attempt to create the query on one line as below I can't compile.
var query = QueryOver.Of<MyType>().Skip(1);
var result = query.Select(myType => myType.Id);
It looks like the code in the first results in query being of type QueryOver< MyType, MyType> while the second results in query being of type QueryOver< MyType>.
It also works if written like this.
var query = QueryOver.Of<MyType>().Select(myType => myType.Id).Skip(1);
Any ideas why the second version fails horribly when the first and third versions work? It seems like odd behavior.
You have a typo in the second version...
var query = QueryOver.Of<MyType().Skip(1);
is missing the >
var query = QueryOver.Of<MyType>().Skip(1);
Not sure if thats what you where looking for.
I'm profiling my application locally (using the Dev server) to get more information about how GAE works. My tests are comparing the common full Entity query and the Projection Query. In my tests both queries do the same query, but the Projection is specified with 2 properties. The test kind has 100 properties, all with the same value for each Entity, with a total of 10 Entities. An image with the Datastore viewer and the Appstats generated data is shown bellow. In the Appstats image, Request 4 is a memcache flush, Request 3 is the test database creation (it was already created, so no costs here), Request 2 is the full Entity query and Request 1 is the projection query.
I'm surprised that both queries resulted in the same amount of reads. My guess is that small and read operations and being reported the same by Appstats. If this is the case, I want to separate them in the reports. That's the queries related functions:
// Full Entity Query
public ReturnCodes doQuery() {
DatastoreService dataStore = DatastoreServiceFactory.getDatastoreService();
for(int i = 0; i < numIters; ++i) {
Filter filter = new FilterPredicate(DBCreation.PROPERTY_NAME_PREFIX + i,
FilterOperator.NOT_EQUAL, i);
Query query = new Query(DBCreation.ENTITY_NAME).setFilter(filter);
PreparedQuery prepQuery = dataStore.prepare(query);
Iterable<Entity> results = prepQuery.asIterable();
for(Entity result : results) {
log.info(result.toString());
}
}
return ReturnCodes.SUCCESS;
}
// Projection Query
public ReturnCodes doQuery() {
DatastoreService dataStore = DatastoreServiceFactory.getDatastoreService();
for(int i = 0; i < numIters; ++i) {
String projectionPropName = DBCreation.PROPERTY_NAME_PREFIX + i;
Filter filter = new FilterPredicate(DBCreation.PROPERTY_NAME_PREFIX + i,
FilterOperator.NOT_EQUAL, i);
Query query = new Query(DBCreation.ENTITY_NAME).setFilter(filter);
query.addProjection(new PropertyProjection(DBCreation.PROPERTY_NAME_PREFIX + 0, Integer.class));
query.addProjection(new PropertyProjection(DBCreation.PROPERTY_NAME_PREFIX + 1, Integer.class));
PreparedQuery prepQuery = dataStore.prepare(query);
Iterable<Entity> results = prepQuery.asIterable();
for(Entity result : results) {
log.info(result.toString());
}
}
return ReturnCodes.SUCCESS;
}
Any ideas?
EDIT: To get a better overview of the problem I have created another test, which do the same query but uses the keys only query instead. For this case, Appstats is correctly showing DATASTORE_SMALL operations in the report. I'm still pretty confused about the behavior of the projection query which should also be reporting DATASTORE_SMALL operations. Please help!
[I wrote the go port of appstats, so this is based on my experience and recollection.]
My guess is this is a bug in appstats, which is a relatively unmaintained program. Projection queries are new, so appstats may not be aware of them, and treats them as normal read queries.
For some background, calculating costs is difficult. For write ops, the cost are returned with the results, as they must be, since the app has no way of knowing what changed (which is where the write costs happen). For reads and small ops, however, there is a formula to calculate the cost. Each appstats implementation (python, java, go) must implement this calculation, including reflection or whatever is needed over the request object to determine what's going on. The APIs for doing this are not entirely obvious, and there's lots of little things, so it's easy to get it wrong, and annoying to get it right.