Filter over Nested Map value's in Spring data mongo query-dsl - mongodb

I am integrating the Spring-Data-mongo with query-dsl, so i need to Generate Q-files for query-dsl queries
Here is my Order POJO:
public class Order {
private List<Map<String,Object>> items;
}
My Q file of Order.java
public class QOrder extends EntityPathBase<Order> {
private static final long serialVersionUID = -682690651L;
public static final QOrder order = new QOrder("order");
public final ListPath<java.util.Map<String, Object>, SimplePath<java.util.Map<String, Object>>> items = this.<java.util.Map<String, Object>, SimplePath<java.util.Map<String, Object>>>createList("items", java.util.Map.class, SimplePath.class, PathInits.DIRECT2);
public QOrder(String variable) {
super(Order.class, forVariable(variable));
}
public QOrder(Path<? extends Order> path) {
super(path.getType(), path.getMetadata());
}
public QOrder(PathMetadata metadata) {
super(Order.class, metadata);
}
}
and sample json's of order is
{
"items": [{
"itemName": "phone",
"quantity": <Integer-Number>
}
]
}
Now i want to retrieve all the order's from mongo for which any Item exist with qunatity 1.
Now i am generating my predicate like below.
"QSensorData.sensorData.data.any().eq(Some-QueryDSL-Expression)".
I am unable to identify what need to pass in eq method for filtering nested map values.

Change Order class to include List attribute where Item contains itemName and quantity fields. Something like
public class Order {
private List<Item> items;
}
public class Item {
private String itemName;
private Integer quantity;
}
Generate Q classes.
Use below query to return all items where there is atleast one item with quantity with 1.
BooleanExpression expression = QOrder.order.items.any().quantity.eq(1);
List<Order> results = repository.findAll(expression);
As noted in the comment to return all filtered items with value 1 you have to use aggregation query.
Something like
Static Imports
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
import static org.springframework.data.mongodb.core.aggregation.ArrayOperators.Filter.filter;
import static org.springframework.data.mongodb.core.aggregation.ComparisonOperators.Eq.valueOf;
Aggregation Query
Aggregation aggregation = newAggregation(
project().and(filter("items")
.as("item")
.by(valueOf(
"item.quantity")
.equalToValue(
1)))
.as("items");
);
List<Order> results = mongoTemplate.aggregate(aggregation, Order.class, Order.class)

Related

How Can I Conditionally Combine Predicates For A JPA Query?

Lets say I have a Book Entity like this:
#Entity
#Table(uniqueConstraints = {
#UniqueConstraint(columnNames = {"title"})
})
class Book {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
Long id;
String title;
String author;
String description;
}
and a repository like this:
#Repository
public interface BookRepository extends JpaRepository<Book, Long>, JpaSpecificationExecutor<Book> {
List<Book> findByAuthor(String author);
Book findByTitle(String title);
List<Book> findByDescriptionContaining(String description);
static Specification<Book> hasTitle(String title) {
return (book, cq, cb) -> cb.equal(book.get("title"), title);
}
static Specification<Book> hasAuthor(String author) {
return (book, cq, cb) -> cb.equal(book.get("author"), author);
}
static Specification<Book> hasDescription(String description) {
return (book, cq, cb) -> cb.like(book.get("description"), "%" + description + "%");
}
}
I can then do a query like this:
repository.findAll(where(hasAuthor("Robert Ludlum")).and(hasTitle("The Bourne Identity")).and(hasDescription("also a film"))).
If I have this in a method with parameters, an empty or null value might be passed.
e.g. a REST API search endpoint that has optional parameters.
In that case I would only want to query by author repository.findAll(where(hasAuthor("Robert Ludlum"))) since adding the other predicates would return no results.
I want to start with a base query that includes everything, then if a parameter is not null add that predicate.
If the author was empty in the above example we wouldn't have a hasAuthor to start the Specification.
How can I conditionally combine the predicates in this way?
You can build your Specification this way.
Specification<Book> spec = Specification.where(null);
if (byAuthor) {
spec = spec.and(hasAuthor("Robert Ludlum"));
}
if (byTitle) {
spec = spec.and(hasTitle("The Bourne Identity"));
}
...
repository.findAll(where(spec));

Spring-boot + Mongodb manual reference not working, can't find my mistake, querying Mongodb works fine

I'm new to Spring-boot and MongoDB. In MongoDB a manual reference between two collections works fine. The mapping in Spring-boot seems not to work. I really don't know what else to check.Below all the relevant details, sorry for the long question.
The reason not to use DBref is because I might need the projections.
The "players" collection has this schema(any other not allowed)
{"_id":{"$oid":"5f56021d61738cc35de79438"},
"name":"Romeo",
"entryDate":{"$date":"2020-08-23T22:00:00.000Z"}}`
The "games" collection has the following schema
{
"_id":{"$oid":"5f5614a361738cc35de7943b"},
"dices":{
"value1":1,
"value2":6
},
"gameScore":1,
"player_id":{"$oid":"5f56021d61738cc35de79438"}
}
The aggregation in MongoDB Compass
[{
$match: {
_id: ObjectId('5f56021d61738cc35de79438')
}
}, {
$lookup: {
from: 'games',
localField: '_id',
foreignField: 'player_id',
as: 'games'
}
}]
yields
In Spring-boot the POJOs are:
#Document(collection = "players")
public class Player {
#Id
private String id;
private String name;
private LocalDate entryDate= LocalDate.now();
private List<Game> game;
public Player(){};
public Player(String name) {
this.name = name;
}
//getters and setters for all properties, including game
}
#Document(collection = "games")
public class Game {
#Id
private String id;
private Dices dices;
private Integer gameScore;
#Field(value = "player_id")
private String playerId;
public Game(){};
public Game(Dices dices) {
this.dices = dices;
}
//getters and setters for all properties
}
public class Dices {
private int value1;
private int value2;
public Dices(){}
public Dices(int value1, int value2) {
this.value1 = value1;
this.value2 = value2;
}
//getters and setters for both properties
In Postman
GET findAll players shows:
[{"id":"5f56021d61738cc35de79438","name":"Romeo","entryDate":[2020,8,24],"game":null},{"id":"5f5602e361738cc35de79439","name":"Julieta","entryDate":[2020,8,24],"game":null},
....]
game is shown because I added also getters and setters for this property, just trying to find the way to properly mapping the games as manual references to players
GET findAll games:
[{"id":"5f5614a361738cc35de7943b","dices":{"value1":1,"value2":6},"gameScore":1,"playerId":"5f56021d61738cc35de79438"},
{"id":"5f5619f561738cc35de7943c","dices":{"value1":2,"value2":5},"gameScore":1,"playerId":"5f5602e361738cc35de79439"},
{"id":"5f561a5461738cc35de7943d","dices":{"value1":3,"value2":3},"gameScore":0,"playerId":"5f56021d61738cc35de79438"},
...]
GET lh:8080/players/5f56021d61738cc35de79438/games
yields an empty array, this is why I assume that the mapping between the collections in Spring-boot fails.
The GamesRepository
#Repository
public interface GameRepository extends MongoRepository<Game, String> {
List<Game> findAll();
List<Game> findGamesByPlayerId(String playerId);
}
The method in the service
#Override
public List<Game> findAllGamesByPlayerId(String playerId) {
Optional<Player> playerDB= playerRepository.findById(playerId);
if(playerDB.isPresent()) {
return gameRepository.findGamesByPlayerId(playerId);
}
else throw new ResourceNotFoundException("Player with id: "+playerId+" does not exist");
}
and the GameController
#GetMapping("/{ID}/games")
public ResponseEntity<List<Game>> getAllGamesByPlayerId (#PathVariable("ID") String playerId){
return ResponseEntity.ok()
.body(gameService.findAllGamesByPlayerId(playerId));
}
Tips are welcome!
Aggregations don't work with MongoRepostory unless you use #DBRef. But using #DBRef is not recommended. What you did in aggregation can be converted into Aggregation pipeline of Spring data.
For that you need to autowired the MongoTemplate
#Autowired
MongoTemplate mongoTemplate;
Then you can convert the aggregation you have written. I haven't tested it, since your aggregation is working, this should work.
public List<Object> test(ObjectId id){
Aggregation aggregation = Aggregation.newAggregation(
match(Criteria.where("_id").is(id)),
lookup("games","_id","player_id","games")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Players.class), Object.class).getMappedResults();
}

MongoTemplate aggregate projection, getting an exception when map array properties

I have two objects and two projection objects like below,
Main Objects
public class MainItem {
private String name;
private List<subItem> subItems;
}
public class subItem {
private String id;
private String groupId;
private String displayName;
private Status status;
}
Projection objects
public class MainItemLight {
private String name;
private List<subItemLight> subItemList;
}
public class subItemLight {
private String id;
private String name;
}
I'm trying to map Main objects to projection objects and return a list of MainItemLight objects. Below is my code,
mongoTemplate.aggregate(
newAggregation(project("name")
.and("subItems").as("subItemList")
.and("subItems.displayName").as("subItemList.name")
),
"MyCollection", MainItemLight.class).getMappedResults();
When I'm trying to map subItems.displayName to subItemList.name I get bellow exception,
Command failed with error 40176 (Location40176): 'Invalid $project
:: caused by :: specification contains two conflicting paths.
Cannot specify both 'subItemList.name' and 'subItemList'
Any idea how to fix this?
You need to do it like this:
db.collection.aggregate([
{
$project: {
name: 1,
subItemList: {
$map: {
input: "$subItems",
as: "item",
in: {
id: "$$item.id",
name: "$$item.displayName"
}
}
}
}
}
])
MongoPlayground
MongoTemplate
Aggregation agg = newAggregation(project("name").and(
VariableOperators.Map.itemsOf("subItems").as("item").andApply(
doc -> new Document()
.append("id", "$$item.id")
.append("name", "$$this.displayName")
)
).as("subItemList"));
Note: The $map implementation is not friendly in Spring Mongo, so wee need to implement it manually.
Source: SpringData mongoDB API for Aggregation $map

Failed to decode property of type BasicDBList with PojoCodecProvider in mongo db

I am tying to load an instance of the class "DataTable" from a mongo database by using the default codec registry (MongoClient.getDefaultCodecRegistry()) and the builder provided by the PojoCodecProvider. I have registered the DataTable class in the codec provider and the object is properly mapped from the database when the records field is null. Nevertheless, I get an error when the records property contains data. Furthermore, I need to have the records field defined as a list of objects with arbitrary attributes. Is it possible to use the default PojoCodecProvider for this purpose? Is there any other alternative?
import com.mongodb.BasicDBList;
import org.bson.types.ObjectId;
import java.util.List;
public class DataTable {
private ObjectId id;
private List<String> fields;
private BasicDBList records;
public ObjectId getId() {
return id;
}
public void setId(ObjectId id) {
this.id = id;
}
public List<String> getFields() {
return fields;
}
public void setFields(List<String> fields) {
this.fields = fields;
}
public BasicDBList getRecords() {
return records;
}
public void setRecords(BasicDBList records) {
this.records = records;
}
}
The exception that I get when load an instance of the DataTable class is the following.
2018-03-21T16:32:04,526 [http-bio-8081-exec-4] ERROR ...service.controllers.BaseController - Failed to decode 'records'. Unable to set value for property 'records' in DataTable
org.bson.codecs.configuration.CodecConfigurationException: Failed to decode 'records'. Unable to set value for property 'records' in DataTable
at org.bson.codecs.pojo.PojoCodecImpl.decodePropertyModel(PojoCodecImpl.java:192) ~[bson-3.6.3.jar:?]
at org.bson.codecs.pojo.PojoCodecImpl.decodeProperties(PojoCodecImpl.java:168) ~[bson-3.6.3.jar:?]
at org.bson.codecs.pojo.PojoCodecImpl.decode(PojoCodecImpl.java:122) ~[bson-3.6.3.jar:?]
at org.bson.codecs.pojo.PojoCodecImpl.decode(PojoCodecImpl.java:126) ~[bson-3.6.3.jar:?]
I get this exception when I try to load an item with the following code
DataTable item = collection.find(eq(new ObjectId(id))).first();
Well, one alternative you can use is Jackson Serialization.
I think something like this would suit you just fine
Document document = collection
.find(eq(new ObjectId(id)))
.first();
String json = document.toJson();
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
DataTable dataTable = mapper.readValue(json, DataTable.class);
See this question converting Document objects in MongoDB 3 to POJOS for reference

Using the $in operator through Morphia - doing it wrong?

I have the following Play Framework entity (using Morphia for persistence) as part of a generic blogging app:
#Entity
public class Comment extends Model {
...
#Reference
#Indexed
public SiteUser commenter;
public static List<Comment> getLastCommentsByUsers(final List<SiteUser> users) {
final Query<Comment> query ds().createQuery(Comment.class);
query.field(commenter).hasAnyOf(users);
return query.asList();
}
}
SiteUser:
#Entity(noClassnameStored=true)
public class SiteUser extends AbstractUser {
public String realName;
}
AbstractUser:
public class AbstractUser extends Model {
#Indexed(value= IndexDirection.DESC, unique = true)
public String emailAddress;
#Required
public String password;
}
The method getLastCommentsByUsers() is supposed to return all comments by the users in the users parameter, but I always get an empty List back. The reason that Commment is a separate collection is to be able to retrieve last X Comments by certain users across their associated Posts, which isn't possible if the Comment is embedded in the Post collection.
Is there something wrong with my query (should I be using something other than hasAnyOf), or is it a problem with the relationship mapping - should I be using ObjectId instead?
I use the in() method with a list or set and its working perfectly. Here's a snippet:
List<String> keywordList;
List<Product> products = Product.find().field("keywords").in(keywordList).asList();
This should work for collection of embedded or references too.
You should use List<Key<SiteUser>> to query:
public static List<Comment> getLastCommentsByUsers(final List<SiteUser> users) {
final Query<Comment> query ds().createQuery(Comment.class);
query.field(commenter).hasAnyOf(toKeys(users)); // convert to keys
return query.asList();
}
public static List<Key<SiteUser>> toKeys(List<SiteUser> users) {
List<Key<SiteUser>> keys = new ArrayList<Key<SiteUser>>();
for(SiteUser user: users) {
keys.add(ds().getMapper().getKey(user));
}
return keys;
}
Or you can just get the keys by:
List<Key<SiteUser>> keys = ds().createQuery(SiteUser.class).query().filter(...).asKeyList();