I am new to MongoDB so please excuse my ignorance.
I have a mongoDB which contains a bunch of documents with something like this
["field": "blah", "version":"1" ...]
["field": "blah", "version":"2" ...]
["field": "blah", "version":"1"....]
["field": "blah1", "version":"10"...]
["field": "blah2", "version":"100"...]
["field": "blah3", "version":"1"....]
["field": "blah3", "version":"2"....]
["field": "blah2", "version":"1"....
I am trying to send a list of queries and fetch all the records as a batch. Is it possible to do so?
List<Docs> fetchDocs(Map<String, String> queries)
{
CriteriaContainer cc=null;
Query<Docs> query = this.mongoDao.createQuery(MyClass.class);
for (Map.Entry<String,String >entry : queries.entrySet())
{
if(cc ==null)
{
cc= query.criteria("Field").equal(entry.getKey()).and(query.criteria("version").equal(entry.getValue()));
}
else
{
cc.or(query.criteria("Field").equal(entry.getKey()).and(query.criteria("version").equal(entry.getValue()));)
}
}
query.and(cc);
List<Docs> doc = query.asList();
return doc;
}
I am not geting the correct list back. I am not sure if I have written this query correctly.
essentially, I want to fetch results or query like this
[{"Field":"blah,"version":1 } $or {"Field":"blah1", "version":10} ]
It should return me a list containing
["field": "blah", "version":"1" ....]
["field": "blah1", "version":"10"....]
Yes, this is definitely possible to do. The final query object will look more like this, however:
{$or: [{"field": "blah,"version":1 }, {"field":"blah1", "version":10} ]}
Notice the array of subqueries for the $or expression.
There are some quirks in Morphia will mean you have to wrap it in an $and clause. So your code would be like this:
Query q = dao.createQuery();
q.and(
q.or(subQuery1),
q.or(subQuery2),
...
);
I ended up fixing my query to this which worked. Wanted to share it with the rest
List<Docs> fetchDocs(Map<String, String> queries){
Criteria [] c = new Criteria[queries.size()];
Query<TUCacheDoc> query0 = this.mongoDao.createQuery(MyClass.class);
int counter=0;
for (Map.Entry<String, String> entry : inMap.entrySet()) {
Query<TUCacheDoc> queryC = this.mongoDao.createQuery(MyClass.class);
String key= entry.getKey();
String val= entry.getValue();
Criteria cC = queryC.criteria("field").equal(key).criteria("version").equal(val);
c[counter]=cC;
counter++;
}
query0.or(c);
List<TUCacheDoc> qresult = query0.asList();
return qresult;
}
Related
I am trying to get result of this MongoDB query on java.
db.fileTree.aggregate([
{
$match: {
"_id": "6062144bb25e4809548ef246",
}
},
{
$unwind: "$children"
},
{
$match: {
"children.fileName": "Test1"
}
},
{
$project: {
"_id": 0,
"fileId": "$children.fileId",
"fileName": "$children.fileName",
"directory": "$children.directory",
}
}
]).pretty()
The query works perfectly fine and it is not showing anything when there is no data. But, the query when executed from java is producing the following error:
com.mongodb.MongoCommandException: Command failed with error 40323 (Location40323): 'A pipeline stage specification object must contain exactly one field.' on server localhost:27017. The full response is {"ok": 0.0, "errmsg": "A pipeline stage specification object must contain exactly one field.", "code": 40323, "codeName": "Location40323"}
ChildFile findChildInParent(String parentId, String fileName) {
BasicDBObject idFilter = new BasicDBObject().append("_id", parentId);
BasicDBObject matchId = new BasicDBObject().append("$match", idFilter);
BasicDBObject unwindChildren = new BasicDBObject().append("$unwind", "$children");
BasicDBObject childNameFilter = new BasicDBObject().append("children.fileName", fileName);
BasicDBObject matchChildName = new BasicDBObject().append("$match", childNameFilter);
BasicDBObject projections = new BasicDBObject()
.append("_id", 0)
.append("fileId", "$children.fileId")
.append("fileName", "$children.fileName")
.append("directory", "$children.directory");
List<ChildFile> childFiles = fileCollection.aggregate(
List.of(matchId, unwindChildren, matchChildName, projections),
ChildFile.class
).into(new ArrayList<>());
return childFiles.size() > 0 ? childFiles.get(0) : null;
}
Am I missing anything here? Any help is really appreciated. Thanks 😃!
There is a typo in your code, you are missing $ for children field
Should be:
BasicDBObject unwindChildren = new BasicDBObject().append("$unwind", "$children")
Instead of:
BasicDBObject unwindChildren = new BasicDBObject().append("$unwind", "children")
Also missing $poject stage:
BasicDBObject projections = new BasicDBObject()
.append("_id", 0)
.append("fileId", "$children.fileId")
.append("fileName", "$children.fileName")
.append("directory", "$children.directory");
BasicDBObject projectionStage = new BasicDBObject().append("$project", projections);
List<ChildFile> childFiles = fileCollection.aggregate(
List.of(matchId, unwindChildren, matchChildName, projectionStage),
ChildFile.class
).into(new ArrayList<>());
I have two collections for example CollectionA and CollectionB both have common filed which is hostname
Collection A :
{
"hostname": "vm01",
"id": "1",
"status": "online",
}
Collection B
{
"hostname": "vm01",
"id": "string",
"installedversion": "string",
}
{
"hostname": "vm02",
"id": "string",
"installedversion": "string",
}
what i want to achieve is when i receive a post message for collection B
I want to check if the record exists in Collection B based on hostname and update all the values. if not insert the new record ( i have read it can be achieved by using upsert -- still looking how to make it work)
I want to check if the hostname is present in Collection A , if not move the record from collection B to another collection which is collection C ( as archive records).ie in the above hostname=vm02 record from collection B should be moved to collectionC
how can i achieve this using springboot mongodb anyhelp is appreciated.The code which i have to save the Collection B is as follows which i want to update to achieve the above desired result
public RscInstalltionStatusDTO save(RscInstalltionStatusDTO rscInstalltionStatusDTO) {
log.debug("Request to save RscInstalltionStatus : {}", rscInstalltionStatusDTO);
RscInstalltionStatus rscInstalltionStatus = rscInstalltionStatusMapper.toEntity(rscInstalltionStatusDTO);
rscInstalltionStatus = rscInstalltionStatusRepository.save(rscInstalltionStatus);
return rscInstalltionStatusMapper.toDto(rscInstalltionStatus);
}
Update 1 : The below works as i expected but I think there should be a better way to do this.
public RscInstalltionStatusDTO save(RscInstalltionStatusDTO rscInstalltionStatusDTO) {
log.debug("Request to save RscInstalltionStatus : {}", rscInstalltionStatusDTO);
RscInstalltionStatus rscInstalltionStatus = rscInstalltionStatusMapper.toEntity(rscInstalltionStatusDTO);
System.out.print(rscInstalltionStatus.getHostname());
Query query = new Query(Criteria.where("hostname").is(rscInstalltionStatus.getHostname()));
Update update = new Update();
update.set("configdownload",rscInstalltionStatus.getConfigdownload());
update.set("rscpkgdownload",rscInstalltionStatus.getRscpkgdownload());
update.set("configextraction",rscInstalltionStatus.getConfigextraction());
update.set("rscpkgextraction",rscInstalltionStatus.getRscpkgextraction());
update.set("rscstartup",rscInstalltionStatus.getRscstartup());
update.set("installedversion",rscInstalltionStatus.getInstalledversion());
mongoTemplate.upsert(query, update,RscInstalltionStatus.class);
rscInstalltionStatus = rscInstalltionStatusRepository.findByHostname(rscInstalltionStatus.getHostname());
return rscInstalltionStatusMapper.toDto(rscInstalltionStatus);
}
Update2 : with the below code i am able to get move the records to another collection
String query = "{$lookup:{ from: \"vmdetails\",let: {rschostname: \"$hostname\"},pipeline:[{$match:{$expr:{$ne :[\"$hostname\",\"$$rschostname\"]}}}],as: \"rscInstall\"}},{$unwind:\"$rscInstall\"},{$project:{\"_id\":0,\"rscInstall\":0}}";
AggregationOperation rscInstalltionStatusTypedAggregation = new CustomProjectAggregationOperation(query);
LookupOperation lookupOperation = LookupOperation.newLookup().from("vmdetails").localField("hostname").foreignField("hostname").as("rscInstall");
UnwindOperation unwindOperation = Aggregation.unwind("$rscInstall");
ProjectionOperation projectionOperation = Aggregation.project("_id","rscInstall").andExclude("_id","rscInstall");
OutOperation outOperation = Aggregation.out("RscInstallArchive");
Aggregation aggregation = Aggregation.newAggregation(rscInstalltionStatusTypedAggregation,unwindOperation,projectionOperation,outOperation);
List<BasicDBObject> results = mongoTemplate.aggregate(aggregation,"rsc_installtion_status",BasicDBObject.class).getMappedResults();
this issue which i have here is it returns multiple records
Found the solution , there may be other best solutions but for me this one worked
create a class customeAggregationGeneration (found in SO answers and extended to match my needs)
public class CustomProjectAggregationOperation implements AggregationOperation {
private String jsonOperation;
public CustomProjectAggregationOperation(String jsonOperation) {
this.jsonOperation = jsonOperation;
}
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext) {
return aggregationOperationContext.getMappedObject(Document.parse(jsonOperation));
}
}
String lookupquery = "{$lookup :{from:\"vmdetails\",localField:\"hostname\",foreignField:\"hostname\"as:\"rscinstall\"}}";
String matchquery = "{ $match: { \"rscinstall\": { $eq: [] } }}";
String projectquery = "{$project:{\"rscinstall\":0}}";
AggregationOperation lookupOpertaion = new CustomProjectAggregationOperation(lookupquery);
AggregationOperation matchOperation = new CustomProjectAggregationOperation(matchquery);
AggregationOperation projectOperation = new CustomProjectAggregationOperation(projectquery);
Aggregation aggregation = Aggregation.newAggregation(lookupOpertaion, matchOperation, projectOperation);
ArrayList<Document> results1 = (ArrayList<Document>) mongoTemplate.aggregate(aggregation, "rsc_installtion_status", Document.class).getRawResults().get("result");
// System.out.println(results1);
for (Document doc : results1) {
// System.out.print(doc.get("_id").toString());
mongoTemplate.insert(doc, "RscInstallArchive");
delete(doc.get("_id").toString());
After reviewing this page, specifically this query
db.scores.find(
{ results: { $elemMatch: { $gte: 80, $lt: 85 } } }
)
I used the following imports
import static com.mongodb.client.model.Filters.and;
import static com.mongodb.client.model.Filters.elemMatch;
import static com.mongodb.client.model.Filters.eq;
import static com.mongodb.client.model.Projections.excludeId;
import static com.mongodb.client.model.Projections.fields;
import static com.mongodb.client.model.Projections.include;
And came up with the following code to perform a similar operation (ARRAY_FIELD_NAME = "myArray")
MongoCollection<Document> collection = mongoDB.getCollection(COLLECTION_NAME);
Bson filters = and(eq("userId", userId), elemMatch(ARRAY_FIELD_NAME, eq("id", id)));
Bson projections = fields(include(ARRAY_FIELD_NAME), excludeId());
List<Document> results = (List<Document>) collection.find(filters).projection(projections).first().get(ARRAY_FIELD_NAME);
if (CollectionUtils.isEmpty(results)) {
return null;
}
if (results.size() > 1) {
throw new ApiException(String.format("Multiple results matched (User ID: %s, Array item ID: %s)", userId, id));
}
return results.get(0);
To filter documents that have the following structure
{
"_id": {
"$oid": "588899721bbabc26865f41cc"
},
"userId": 55,
"myArray": [
{
"id": "5888998e1bbabc26865f41d2",
"title": "ABC"
},
{
"id": "5888aaf41bbabc3200e252aa",
"title": "ABC"
}
]
}
But instead of getting a single or no item from the myArray field, I always get both items !
The only code that worked for me is the following
MongoCollection<Document> collection = mongoDB.getCollection(COLLECTION_NAME);
List<Bson> aggregationFlags = new ArrayList<>();
aggregationFlags.add(new Document("$unwind", "$" + ARRAY_FIELD_NAME));
aggregationFlags.add(new Document("$match", new Document("userId", userId).append(ARRAY_FIELD_NAME + ".id", id)));
aggregationFlags.add(new Document("$project", new Document("_id", 0).append(ARRAY_FIELD_NAME, "$" + ARRAY_FIELD_NAME)));
return (Document) collection.aggregate(aggregationFlags).first().get(ARRAY_FIELD_NAME);
So why does the first piece of code that should behave the same as the query shown at the beginning of the question, not filter results as expected ?
I do not need to "aggregate" results, I need to "filter" them using the user ID and array item id.
You need to use $elemMatch(projection). Something like below should work.
import static com.mongodb.client.model.Projections.elemMatch;
Bson filters = and(eq("userId", userId));
Bson projections = fields(elemMatch(ARRAY_FIELD_NAME, eq("id", id)), excludeId());
I want to pagination with Spring Data Mongo.From docs spring data mongo can do :
public interface TwitterRepository extends MongoRepository<Twitter, String> {
List<Twitter> findByNameIn(List<String> names, Pageable pageable);
}
If Twitter Document Object like this:
#Document
public class Twitter {
String name;
#DBRef
List<Comment> comments
}
Does spring data mongo support pagination with comments?
Note: The code specified is not tested, it will just serve as a pointer for you
The following mongo query limits the the array size to be returned:
db.Twitter.find( {}, { comments: { $slice: 6 } } )
The above mechanism can be used to enforce pagination like so:
db.Twitter.find( {}, { comments: { $slice: [skip, limit] } } )
You can try by annotating your method
#Query(value="{ 'name' : {'$in': ?0} }", fields="{ 'comments': { '$slice': [?1,?2] } }")
List<Twitter> findByNameIn(List<String> names, int skip, int limit);
}
You can specify that in your query like so:
Query query = new Query();
query.fields().slice("comments", 1, 1);
mongoTemplate.find(query, DocumentClass.class);
or you can try and execute the command directly using:
mongoTemplate.executeCommand("db.Twitter.find( {}, { comments: { $slice: [skip, limit] } } )")
General Pagination Mechanisms:
General Pagination mechanisms only work at the document level, examples of which are given below.
For them you will have to manually splice the returned comments at the application level.
If you using the MongoTemplate class (Spring-Data Docs) then:
Use org.springframework.data.mongodb.core.query.Query class's skip() and limit() method to perform pagination
Query query = new Query();
query.limit(10);
query.skip(10);
mongoTemplate.find(query, DocumentClass.class);
If you are using Repository (Spring-Data-Reposioty) then use PagingAndSortingRepository
I am using elastic4s 1.5.10 and trying to build up a query that I prepared on elasticsarch REST endpoint. Now trying to rewrite to elastic4s dsl.
POST /index_name/type/_search
{
"_source":{"include":["name","surname"]},
"query": {
"bool": {
"must": [
{
"more_like_this": {
"fields": ["desc"],
"ids": ["472825948"],
"min_term_freq":0,
"max_query_terms":200
}
},
{
"match": {"city": "London"}
},
{
"match": {"operation": "add"}
}
]
}
}
}
The goal of this query is get similar items to 472825948 which has the same operation (add) in the same city (London).
My attempt in elastic4s follows:
es_client.execute{
search in s"$storage_folder/${typ.name()}" query bool(
must(
morelike id adId in s"$storage_folder/${typ.name()}" fields("desc") minTermFreq(0) maxQueryTerms(200),
matchQuery(field="city",value = "London"),
matchQuery(field="operation",value = "add"))
)sourceInclude("name","surname")
}
}
"morelike" doesn't work in this context. Either the query as it is in raw json doesn't make sense or elastic4s doesn't support this or ...
Could someone help?
Thx
Just for completeness adding response I got here as well.
morelikee is for a request type, morelikeThisQuery is the correct form of the query. So the resulted query should look like as follows
es_client.execute{
search in s"$storage_folder/${typ.name()}" query bool(
must(
morelikeThisQuery fields("desc") ids(adId) minTermFreq(0) maxQueryTerms(200),
matchQuery(field="city",value = "London"),
matchQuery(field="operation",value = "add"))
)sourceInclude("name","surname")
}
}