I'm trying to query MongoDB for Just idProfil and hashtags list from profileTwitter and bellow is the object:
{
"idProfil": "5ded2abae1692808b799b239",
"tweets": [
{
"idTweet": "5ded2dffe1692808b799b241",
"datePublication": "2019-12-08T16:58:59.702+0000",
"hashtags": [
{
"idHashtag": "5ded2c64e1692808b799b23c",
"label": "recipes "
},
{
"idHashtag": "5ded2c71e1692808b799b23d",
"label": "delicious "
},
{
"idHashtag": "5ded2c7ce1692808b799b23e",
"label": "foodrecipes"
},
{
"idHashtag": "5ded2c84e1692808b799b23f",
"label": "canada "
},
{
"idHashtag": "5ded2c8de1692808b799b240",
"label": "Usa"
},
{
"idHashtag": "5dee65d7e39e962d44a31c40",
"label": "food"
},
{
"idHashtag": "5dee65c8e39e962d44a31c3f",
"label": "cooking"
}
]
}
}
So my question is how to do this using spring boot ?
This example based on data you have provided. Your question is not clear. Can you please provide more details on it.
List<AggregationOperation> stages = new ArrayList<>();
ProjectOperation projectOperation = project("idProfil").and("$tweets.hashtags").as("hashtags");
stages.add(projectOperation);
AggregationResults<ResultDTO> result = mongoOperation.aggregate(newAggregation(stages),
"profileTwitter", ResultDTO.class);
public class ResultDTO {
private String idProfil;
private List<HashtagDTO> hashtags;
//getter setter
}
public class HashtagDTO {
private String idHashtag;
private String label;
//getter setter
}
Related
I'm starting learn spring data mongodb. Currently, I try to implement multiple condition query with Spring Data MongoDB.
First I have a sample collection
{
"_id": "df05532f-8893-4802-8710-ab92056c9c77",
"objectId": {
"$numberLong": "1"
},
"creator": {
"_id": {
"$numberLong": "3"
},
"username": "magic"
},
"content": "7878787887",
"attachments": [],
"pinned": false,
"histories": [
{
"content": "new new comment 2222",
"createdAt": {
"$date": {
"$numberLong": "1664970753576"
}
}
},
{
"content": "update me",
"createdAt": {
"$date": {
"$numberLong": "1664970753691"
}
}
},
{
"content": "update me3333",
"createdAt": {
"$date": {
"$numberLong": "1664970753734"
}
}
},
{
"content": "44444455666",
"createdAt": {
"$date": {
"$numberLong": "1664970753740"
}
}
}
],
"contentUpdateAt": {
"$date": {
"$numberLong": "1664970753745"
}
},
"createdAt": {
"$date": {
"$numberLong": "1664970753576"
}
},
"lastModifiedAt": {
"$date": {
"$numberLong": "1664970753772"
}
}
}
I'm trying to find a partial document when the array histories has one element that matched with content equals 44444455666
With MongoDB CLI, I completed build the query like
db.ticket_detail_comments.find(
{
histories: {
$elemMatch: {
content: "44444455666"
}
},
_id : "df05532f-8893-4802-8710-ab92056c9c77"
},
{ "histories.$": 1, name: 1 }
)
The result was exactly what I expected
{
"_id": "df05532f-8893-4802-8710-ab92056c9c77",
"histories": [
{
"content": "44444455666",
"createdAt": {
"$date": {
"$numberLong": "1664970753740"
}
}
}
]
}
When I tried to build query with Spring Data MongoDB (using MongoDbTemplate) I can't find any way to build query look like in console. I tried addOperation but seem not work,
I tried to build 2 separated Criteria (one, two) that match with 2 condition I expected, then combine them with andOperation like new Criteria().andOperator(one).andOperator(two);. The unhappy part is the query generated is so different
db.ticket_detail_comments.find(
{
"$and": [
{
histories: {
$elemMatch: {
content: "44444455666"
}
},
_id : "df05532f-8893-4802-8710-ab92056c9c77"
},
{ "histories.$": 1, name: 1 }
]
}
)
With this query, I always get a null value. Can anyone help me with that? What is correct Criteria syntax I should use?
Thanks for reading.
UPDATE
I created a small demo, let's see
Data in Mongo Collection look like
{
"_id": "dad9db98-241b-40fa-aab4-af1671f97631",
"data": "demo",
"children": [
{
"name": "tim",
"age": 19
},
{
"name": "alex",
"age": 18
}
],
"_class": "com.example.demo.Data"
}
MONGOCLI
db.data.find(
{
children: {
$elemMatch: {
name: "alex"
}
},
_id : "dad9db98-241b-40fa-aab4-af1671f97631"
},
{ "children.$": 1}
)
=> Result:
{ _id: 'dad9db98-241b-40fa-aab4-af1671f97631',
children: [ { name: 'alex', age: 18 } ] }
JAVA CODE
package com.example.demo;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.NoArgsConstructor;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import javax.annotation.PostConstruct;
import java.util.List;
import java.util.UUID;
#SpringBootApplication
public class DemoApplication {
private final MongoTemplate template;
public DemoApplication(MongoTemplate template) {
this.template = template;
}
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
#PostConstruct
public void init() {
Data.Child child1 = Data.Child.builder()
.name("alex")
.age(18)
.build();
Data.Child child2 = Data.Child.builder()
.name("tim")
.age(19)
.build();
final String id = UUID.randomUUID().toString();
Data data = Data.builder()
.id(id)
.data("demo")
.children(List.of(child2, child1))
.build();
template.save(data, "data");
Query query = Query.query(
Criteria.where("_id").is(id).and("children")
.elemMatch(Criteria.where("name").is("alex"))
);
query.fields().position("children", 1);
var r = template.findOne(query, Data.class, "data");
System.out.printf("DEBUG");
}
}
#lombok.Data
#AllArgsConstructor
#NoArgsConstructor
#Builder
class Data {
private String id;
private String data;
private List<Child> children;
#lombok.Data
#AllArgsConstructor
#NoArgsConstructor
#Builder
static class Child {
private String name;
private Integer age;
}
}
=> Result: Data Object with 2 elements in children list.
But what I expected is Data Object has one element in children list like resut from CLI
I'm developing a method in Spring-Data-MongoDB that use $facet,$addFields and $function but it return null.
This is the aggregation in MongoDB
db.utenti.aggregate([
{
$facet:
{
"aggregation": [
{
$match: { age: { $gt: 18 } }
},
{
$addFields:{
max: {
$function: {
body: function(name, scores) {
let max = Math.max(...scores);
return `Hello ${name}. Your max score is ${max}.`
},
args: [ "$name", "$scores"],
lang: "js"
}
}
}
}
]
}
}
]).pretty()
This the expected result
{
"aggregation" : [
{
"_id" : ObjectId("62c5f16b39b3d635ab6bf60d"),
"username" : "james34",
"name" : "james",
"role" : [
"user",
"specialuser"
],
"age" : 34,
"scores" : [
10,
9,
10
],
"_class" : "com.project.ecommercemongo.model.User",
"max" : "Hello james. Your max score is 10."
},
{
"_id" : ObjectId("62c5f1b839b3d635ab6bf60e"),
"username" : "elizabeth54",
"name" : "elizabeth",
"role" : [
"user",
"specialuser"
],
"age" : 54,
"scores" : [
10,
10,
10
],
"_class" : "com.project.ecommercemongo.model.User",
"max" : "Hello elizabeth. Your max score is 10."
},
{
"_id" : ObjectId("62c5f1f139b3d635ab6bf60f"),
"username" : "frank50",
"name" : "frank",
"role" : [
"user"
],
"age" : 50,
"scores" : [
10,
10,
10
],
"_class" : "com.project.ecommercemongo.model.User",
"max" : "Hello frank. Your max score is 10."
},
{
"_id" : ObjectId("62c5f27a39b3d635ab6bf610"),
"username" : "john26",
"name" : "john",
"role" : [
"user"
],
"age" : 26,
"scores" : [
8,
8,
10
],
"_class" : "com.project.ecommercemongo.model.User",
"max" : "Hello john. Your max score is 10."
}
]
}
This is the result I get
[
{
"_id": null,
"username": null,
"name": null,
"role": null,
"age": null,
"scores": null
}
]
this is the method that call facet addFields and function,but it don't add the field "max" and returns null
public List<User> aggregation1() {
List<Object> bl = new LinkedList<Object>();
bl.add("$name");
bl.add("$scores");
ScriptOperators.Function function= Function.function("function(name, scores) {let max = Math.max(...scores); return `Hello ${name}. Your max score is ${max}`}").args(bl).lang("js");//`Hello ${name}. Your max score is ${max}.`}");
System.out.println(function.toDocument());
FacetOperation facetOperation1 = Aggregation.facet(Aggregation.match(Criteria.where("age").gte(18)),Aggregation.addFields().addFieldWithValue("max", function).build()).as("aggregation");
Aggregation agg = Aggregation.newAggregation(facetOperation1);
return mongoTemplate.aggregate(agg, "utenti",User.class).getMappedResults();
}
And this is the User class
#Document(collection="utenti")
//#Sharded(shardKey = { "country", "userId" })
public class User {
#Id
private String _id;
private String username;
private String name;
private String[] role;
private Integer age;
private Integer[] scores;
public User() {
super();
}
public String get_id() {
return _id;
}
public void set_id(String _id) {
this._id = _id;
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String[] getRole() {
return role;
}
public void setRole(String[] role) {
this.role = role;
}
public Integer[] getScores() {
return scores;
}
public void setScores(Integer[] scores) {
this.scores = scores;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
}
how can i put all the users into the facet and add a field ? Thank you
Update
Now it work, I had to map the output with the aggregation fields
public List<MaxFacet> aggregation1() {
List<Object> bl = new LinkedList<Object>();
bl.add("$name");
bl.add("$scores");
ScriptOperators.Function function = Function.function(
"function(name, scores) {let max = Math.max(...scores); return `Hello ${name}. Your max score is ${max}.`}")
.args(bl).lang("js");// `Hello ${name}. Your max score is ${max}.`}");
System.out.println(function.toDocument());
FacetOperation facetOperation = Aggregation.facet().and(match(Criteria.where("age").gte(18)),
Aggregation.addFields().addFieldWithValue("max", function).build()).as("aggregation");
Aggregation agg = Aggregation.newAggregation(facetOperation);
return mongoOperations.aggregate(agg, mongoTemplate.getCollectionName(User.class), MaxFacet.class)
.getMappedResults();
}
public class MaxFacet {
private List<UserOut> aggregation;
public List<UserOut> getAggregation() {
return aggregation;
}
public void setAggregation(List<UserOut> facet1) {
this.aggregation = facet1;
}
public MaxFacet() {
// TODO Auto-generated constructor stub
}
}
#JsonPropertyOrder({"id","username","name","age","scores","max"})
public class UserOut {
#JsonProperty(value="id")
private String id;
private String username;
private String name;
private String []scores;
private String max;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String[] getScores() {
return scores;
}
public void setScores(String[] scores) {
this.scores = scores;
}
public String getMax() {
return max;
}
public void setMax(String value) {
this.max= value;
}
}
[
{
"aggregation": [
{
"id": "62c5f16b39b3d635ab6bf60d",
"username": "james34",
"name": "james",
"scores": [
"10",
"9",
"10"
],
"max": "Hello james. Your max score is 10."
},
{
"id": "62c5f1b839b3d635ab6bf60e",
"username": "elizabeth54",
"name": "elizabeth",
"scores": [
"10",
"10",
"10"
],
"max": "Hello elizabeth. Your max score is 10."
},
{
"id": "62c5f1f139b3d635ab6bf60f",
"username": "frank50",
"name": "frank",
"scores": [
"10",
"10",
"10"
],
"max": "Hello frank. Your max score is 10."
},
{
"id": "62c5f27a39b3d635ab6bf610",
"username": "john26",
"name": "john",
"scores": [
"8",
"8",
"10"
],
"max": "Hello john. Your max score is 10."
}
]
}
]
Alternative soluition: You can simplify your code this way
1- Use inheritance mecanism for the UserOut class
public class UserOut extends User {
private String max;
public String getMax() {
return max;
}
public void setMax(String value) {
this.max= value;
}
}
2- For the complex pipelines, the custom AggregationOperation implementation is friendlier than Spring Data builder.
AggregationOperation facetOperation = ao ->
new Document("$facet",
new Document("aggregation", Arrays.asList(
new Document("$match", new Document("age", new Document("$gt", 18))),
new Document("$addFields",
new Document("max",
new Document("$function",
new Document("body", "function(name, scores) {"+
" let max = Math.max(...scores);"+
" return `Hello ${name}. Your max score is ${max}.`"+
"}")
.append("args", Arrays.asList("$name", "$scores"))
.append("lang", "js")
)
)
)
))
);
Aggregation agg = Aggregation.newAggregation(facetOperation);
return mongoOperations
.aggregate(agg, mongoTemplate.getCollectionName(User.class), MaxFacet.class)
.getMappedResults();
Alternatively, just parse your shell JSON query this way (triple quotes starting from Java 13):
Document.parse("""
{
"$facet":
{
"aggregation": [
{
"$match": { "age": { "$gt": 18 } }
},
{
"$addFields":{
"max": {
"$function": {
"body": function(name, scores) {
let max = Math.max(...scores);
return `Hello ${name}. Your max score is ${max}.`
},
"args": [ "$name", "$scores"],
"lang": "js"
}
}
}
}
]
}
}
""")
How to fetch the document by array of value from MONGODB using SPRING DATA.?
My document
{
"id": "5b18fdef89d67e272025f2e3",
"date": "2018-05-10 11:31:37",
"active": true,
"obsolete": false,
"tenant": {
"id": "5ad847e54925fb0fa4424e1a"
},
"plan": {
"id": "5ad7115f7b4152204c86fce6"
},
"log": [
"5b18fdef89d67e272025f2e4"
],
"lineItem": [
"5b18fdef89d67e272025f2e5",
"5b18fdef89d67e2720259899",
"5b18fdef89d67e272025sd5s"
]
}
{
"id": "5b18fdef89d67e272025f232",
"date": "2018-05-12 11:31:37",
"active": true,
"obsolete": false,
"tenant": {
"id": "5ad847e54925fb0fa4424e1a"
},
"plan": {
"id": "5ad7115f7b4152204c86fce6"
},
"log": [
"5b18fdef89d67e272025f23434"
],
"lineItem": [
"5b18fdef89d67e272025f111",
"5b18fdef89d67e2720259222",
"5b18fdef89d67e272025s333"
]
}
I want filter the document by lineItem array. If I give the lineItem value "5b18fdef89d67e2720259222" it will be return document which holds the same lineitem.
Model class
#Document(collection = "trn_inventory")
public class Inventory {
#Id
private String id;
private String date;
private List<String> lineitem;
private String tenant, plan;
private List<String> log;
private boolean active, obsolete;
//getters and setters
}
Repository
#Repository
public interface InventoryRep extends MongoRepository<Inventory, String> {
public List<Inventory> findByLineitemIn(String lineitem);
}
I have done the mistake in Repository. After spend the long time, I found it myself (stupid coder). The problem is parameter of the findByLineitemIn method is String that should be List type.
Here is the corrected Repository...
#Repository
public interface InventoryRep extends MongoRepository<Inventory, String> {
public List<Inventory> findByLineitemIn(List<String> lineitem);
}
How foolish is this.. Isn't it? I think this will helpful for some other coder like me. ;)
I am new to drools and the kie-server. I got problem about how to insert new facts in working memory by fired rule RHS(then action) then activate other rules by these dynamic-insered facts.
What I expect is when BaseFeature insert into Working memory from outside and activate RuleOne, inside the RuleOne "then" it will insert the new fact RuleResult, and expected to activate the RuleTwo, but it doesn't activate RuleTwo, just RuleOne was activated, is that something I doing it wrong?
Here I have two rules and a set of facts:
FACTS definition:
public class AppResult implements java.io.Serializable
{
static final long serialVersionUID = 1L;
private java.lang.String key;
private java.lang.String value;
//getters and setters ...
}
public class BaseFeature implements java.io.Serializable
{
static final long serialVersionUID = 1L;
private int age;
//getters and setters ...
}
public class RuleResult implements java.io.Serializable
{
static final long serialVersionUID = 1L;
private java.lang.String ruleName;
private java.lang.Long score;
private boolean state;
//getters and setters ...
}
Rule definition:
rule "RuleOne"
dialect "mvel"
salience 100
no-loop true
lock-on-active true
when
$b : BaseFeature( age < 22 || age > 40 )
then
RuleResult $r = new RuleResult();
$r.setRuleName( "RuleOne" );
$r.setState( false );
insertLogical( $r );
end
rule "RuleTwo"
dialect "mvel"
no-loop false
lock-on-active true
salience 10
when
$r : RuleResult( ruleName == "RuleOne" , state == false )
$a : AppResult( )
then
$a.setKey( "PASS" );
$a.setValue( "false" );
end
how to achieve fired-rules insert new fact and fired other rules dynamic?
thank in advance!
I am using kie-server(6.5 Final) to test, and the POST and Reponse as below:
POST:
{
"lookup": "RuleChainTestStateless",
"commands": [
{
"insert": {
"return-object": false,
"object": {
"com.qf.rulechaintest.BaseFeature": {
"age": "10"
}
}
}
},
{
"insert": {
"return-object": true,
"out-identifier": "AppResult",
"object": {
"com.qf.rulechaintest.AppResult": {
}
}
}
},
{
"fire-all-rules": ""
},
{
"get-objects":{
"out-identifier":"allFactsInWrokingMemory"
}
}
]
}
REPONSE:
{
"type": "SUCCESS",
"msg": "Container RuleChainTest1.1 successfully called.",
"result": {
"execution-results": {
"results": [
{
"key": "",
"value": 1
},
{
"key": "AppResult",
"value": {
"com.qf.rulechaintest.AppResult": {
"key": null,
"value": null
}
}
},
{
"key": "allFactsInWrokingMemory",
"value": [
{
"com.qf.rulechaintest.BaseFeature": {
"age": 10
}
},
{
"com.qf.rulechaintest.AppResult": {
"key": null,
"value": null
}
},
{
"com.qf.rulechaintest.RuleResult": {
"ruleName": "RuleOne",
"score": null,
"state": false
}
}
]
}
],
"facts": [
{
"key": "AppResult",
"value": {
"org.drools.core.common.DefaultFactHandle": {
"external-form": "0:2:1542374590:1542374590:2:DEFAULT:NON_TRAIT:com.qf.rulechaintest.AppResult"
}
}
}
]
}
}
}
Notice that the reponse:
"com.qf.rulechaintest.AppResult": {
"key": null,
"value": null
}
what it expected to be:
"com.qf.rulechaintest.AppResult": {
"key": "PASS",
"value": "false"
}
If you don't know the purpose of these rule attribute you should read the documentation. Then, I think, you'll come to realize that you can remove them.
salience 100 or 10
no-loop true or false
lock-on-active true
I agree with laune that you should read what the rule attributes mean (see ere in the docs [1]). I think it is because you have lock-on-active on the rules. Please see the docs that I linked.
[1] https://docs.jboss.org/drools/release/6.5.0.Final/drools-docs/html/ch08.html#d0e9196
Try using update function
"
then
RuleResult $r = new RuleResult();
$r.setRuleName( "RuleOne" );
$r.setState( false );
update ( $r );
"
I am trying to create an elastic search index using the following Java code
public class MappingCreator {
static Logger log = Logger.getLogger(MappingCreator.class.getName());
final static String indexName = "indxyz";
final static String typeName = "indtype";
final static String mappingFileName = "customMapping.json";
final static String clusterName = "elasticsearch";
final static String hostName = "localhost";
public static void main(String args[]) throws IOException
{
MappingCreator mapCreator = new MappingCreator();
Client myESclient = getClient();
IndicesExistsResponse res = myESclient.admin().indices().prepareExists(indexName).execute().actionGet();
if (res.isExists()) {
log.warn("Index "+indexName +" already exists. Will be deleted");
final DeleteIndexRequestBuilder deleteIndexBuilder = myESclient.admin().indices().prepareDelete(indexName);
deleteIndexBuilder.execute().actionGet();
}
final CreateIndexRequestBuilder createIndexBuilder = myESclient.admin().indices().prepareCreate(indexName)
.addMapping(typeName, mapCreator.getIndexFieldMapping());
CreateIndexResponse createIndexResponse = createIndexBuilder.execute().actionGet();
log.debug("Created mapping "+createIndexResponse.toString());
myESclient.close();
}
private String getIndexFieldMapping() throws IOException {
return IOUtils.toString(getClass().getClassLoader().getResourceAsStream(mappingFileName));
}
private static Client getClient() {
TransportClient transportClient = null;
try
{
Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", clusterName).build();
transportClient = new TransportClient(settings);
transportClient = transportClient.addTransportAddress(new InetSocketTransportAddress(hostName, 9300));
}
catch (Exception e)
{
log.error("Error in MappingCreator creating Elastic Search Client\n"
+ "Message "+e.getMessage()+"\n"
+ "StackTrace "+e.getStackTrace()
);
}
return (Client) transportClient;
}
}
customMapping.json looks like this
{
"properties": {
"popkDate": {
"type":"date",
"format": "yyyy-MM-dd 'T' HH:mm:ss.SSZ"
},
"popk": {
"type": "string"
}
}
}
now, after creating the mapping, if i run the following in the elasticsearch marvel plugin
get indxyz/
i get
{
"indxyz": {
"aliases": {},
"mappings": {
"indtype": {
"properties": {
"popkDate": {
"type": "date",
"format": "yyyy-MM-dd 'T' HH:mm:ss.SSZ"
},
"popk": {
"type": "string"
}
}
}
}
which is accurate.
now when i try to ingest this data
{
"popkDate":"2001-01-01T235959.00-0400",
"popk":"cuinbcwenvubn"
}
for some reason, the popkDate field is indexed as String, instead of Date. Now, if i again look at
get indxyz/
this time i get,
{
"indxyz": {
"aliases": {},
"mappings": {
"indtype": {
"properties": {
"popkDate": {
"type": "date",
"format": "yyyy-MM-dd 'T' HH:mm:ss.SSZ"
},
"popk": {
"type": "string"
}
}
},
"indtype": {
"properties": {
"popkDate": {
"type": "string"
},
"popk": {
"type": "string"
}
}
}
}
why was this duplicate mapping created ? for the same type ? Note that if i create the mapping without the JAVA api, i.e. through curl commands, then this behavior is not observed and the popkDate field is indexed as expected. Why would this be ?
Please advise.