I am trying to add a Map field in my oracle nosqltable (in the example given here https://docs.oracle.com/en/database/other-databases/nosql-database/21.1/java-driver-table/accessing-nosql-using-sdf.html) but when saving its not getting saved properly by Spring data.
Customer.java
#NosqlTable(storageGB = 1, writeUnits = 10, readUnits = 10)
public class Customer {
#NosqlId(generated = true)
long customerId;
String firstName;
String lastName;
Map hashMap;
Date createdAt;
#Override
public String toString() {
return "Customer{" +
"customerId=" + customerId +
", firstName='" + firstName + '\'' +
", lastName='" + lastName + '\'' +
", createdAt='" + createdAt + '\'' +
", hashMap='" + hashMap + '\'' +
'}';
}
}
CustomerRepostory.java
import com.oracle.nosql.spring.data.repository.NosqlRepository;
public interface CustomerRepository
extends NosqlRepository<Customer, Long>
{
Iterable<Customer> findByLastName(String lastname);
}
When I call the following code to create a customer row :
Customer s1 = new Customer();
s1.firstName = "John";
s1.lastName = "Doe";
HashMap s1Map = new HashMap() ;
s1Map.put("name", "myMap") ;
s1Map.put("use", true);
s1.hashMap = s1Map;
repo.save(s1);
It gets saved as
{
"createdAt": null,
"firstName": "John",
"hashMap": {
"entrySet": null,
"keySet": null,
"loadFactor": 0.75,
"modCount": 2,
"size": 2,
"table": [
null,
null,
null,
null,
null,
null,
{
"hash": 116102,
"key": "use",
"next": null,
"value": true
},
null,
{
"hash": 3373752,
"key": "name",
"next": null,
"value": "myMap"
},
null,
null,
null,
null,
null,
null,
null
],
"threshold": 12,
"values": null
},
"lastName": "Doe"
}
Can someone please help me with correct data type to use for nosql map ?
Map (java.util.Map) is not currently supported, but it is a key roadmap item.
see here https://github.com/oracle/nosql-spring-sdk/issues/18
You can find the current Mapping Between Java and NoSQL JSON Types in the documentation - https://docs.oracle.com/en/database/other-databases/nosql-database/22.2/springsdk/persistence-model.html
In the meantime, it is possible to use the class oracle.nosql.driver.values.MapValue. Here is an example
import com.oracle.nosql.spring.data.core.mapping.NosqlId;
import oracle.nosql.driver.values.MapValue ;
#NosqlTable(storageGB = 1, writeUnits = 10, readUnits = 10)
public class Customer {
#NosqlId(generated = true)
long customerId;
String firstName;
String lastName;
MapValue map;
#Override
public String toString() {
return "Customer{" +
"customerId=" + customerId +
", firstName='" + firstName + '\'' +
", lastName='" + lastName + '\'' +
", map='" + map + '\'' +
'}';
}
}
Here is an example of a call to create a customer row - using multiple types, including other nosql Classes oracle.nosql.driver.values.ArrayValue . As you can see the syntax is very similar to java.util.Map. When java.util.Map will be supported you can migrate easily
Customer c2 = new Customer();
c2.firstName = "John";
c2.lastName = "Josh";
c2.map = new MapValue();
c2.map.put("number field", 1);
c2.map.put("string_field", "string value");
c2.map.put("boolean_field", true);
ArrayValue arrayValue = new ArrayValue();
arrayValue.add(100);
arrayValue.add("102");
arrayValue.add(true);
c2.map.put("json-field", arrayValue);
repo.save(c2);
System.out.println("\nsaved: " + c2);
System.out.println("\nfindAll:");
Iterable<Customer> customers = repo.findAll();
for (Customer s : customers) {
System.out.println(" Customer: " + s);
}
System.out.println("\nfindByLastName: Josh");
customers = repo.findByLastName("Josh");
for (Customer s : customers) {
System.out.println(" Customer: " + s);
}
Here are the outputs - application
saved: Customer{customerId=10, firstName='John', lastName='Doe', map='null'}
saved: Customer{customerId=11, firstName='John', lastName='Smith', map='null'}
saved: Customer{customerId=12, firstName='John', lastName='Josh', map='{"number field":1,"json-field":[100,"102",true],"string_field":"string value"}'}
findAll:
Customer: Customer{customerId=12, firstName='John', lastName='Josh', map='{"json-field":[100,"102",true],"number field":1,"string_field":"string value"}'}
Customer: Customer{customerId=10, firstName='John', lastName='Doe', map='null'}
Customer: Customer{customerId=11, firstName='John', lastName='Smith', map='null'}
findByLastName: Josh
Customer: Customer{customerId=12, firstName='John', lastName='Josh', map='{"json-field":[100,"102",true],"number field":1,"string_field":"string value"}'}
reading using the SQL for Oracle NoSQL Database Shell - if Cloud, you can use the OCI Console
sql-> mode json -pretty
Query output mode is pretty JSON
sql-> select * from customer;
{
"customerId" : 13,
"kv_json_" : {
"firstName" : "John",
"lastName" : "Doe",
"map" : null
}
}
{
"customerId" : 14,
"kv_json_" : {
"firstName" : "John",
"lastName" : "Smith",
"map" : null
}
}
{
"customerId" : 15,
"kv_json_" : {
"firstName" : "John",
"lastName" : "Josh",
"map" : {
"boolean_field" : true,
"json-field" : [100, "102", true],
"number field" : 1,
"string_field" : "string value"
}
}
}
In order to run this example, you need to apply a fix, Otherwise, you will have the following error
Caused by: java.lang.IllegalArgumentException: Entity must not be null!
A fix will be published soon on this GitHub repository https://github.com/oracle/nosql-spring-sdk. Again more information here https://github.com/oracle/nosql-spring-sdk/issues/18
to learn more on MapValue
https://oracle.github.io/nosql-java-sdk/oracle/nosql/driver/values/MapValue.html
https://oracle.github.io/nosql-java-sdk/oracle/nosql/driver/values/FieldValue.html
Related
I am saving a polymorphic model using mongoDB, and the data is saved perfectly, for example.
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.EXISTING_PROPERTY, property = "type", visible = true)
#JsonSubTypes(
JsonSubTypes.Type(value = B::class, name = "test_a"),
JsonSubTypes.Type(value = C::class, name = "test_b")
)
data class Base(
val list:List<A>? = null
)
open class A(
val type: String? = null
)
data class B : A(
val value: String? = null
)
data class C : A(
val values: List<String>? = null
)
the data is reflected in the database as follows
{
"list":[
{
"type": "type a",
"value": "test"
},
{
"type": "type b",
"values": [
"test 1",
"test 2"
]
}
]
}
but when i try to get the data from mongo db it returns it but without the attributes of child classes B and C.
{
"list":[
{
"type": "type a"
},
{
"type": "type b"
}
]
}
I am getting the data using mongoRepository with this annotation.
#Query("{'id' : ?0 }")
fun findById(#Param("id") id: String): Base?
Do you know what could be happening?
I'm developing a method in Spring-Data-MongoDB that use $facet,$addFields and $function but it return null.
This is the aggregation in MongoDB
db.utenti.aggregate([
{
$facet:
{
"aggregation": [
{
$match: { age: { $gt: 18 } }
},
{
$addFields:{
max: {
$function: {
body: function(name, scores) {
let max = Math.max(...scores);
return `Hello ${name}. Your max score is ${max}.`
},
args: [ "$name", "$scores"],
lang: "js"
}
}
}
}
]
}
}
]).pretty()
This the expected result
{
"aggregation" : [
{
"_id" : ObjectId("62c5f16b39b3d635ab6bf60d"),
"username" : "james34",
"name" : "james",
"role" : [
"user",
"specialuser"
],
"age" : 34,
"scores" : [
10,
9,
10
],
"_class" : "com.project.ecommercemongo.model.User",
"max" : "Hello james. Your max score is 10."
},
{
"_id" : ObjectId("62c5f1b839b3d635ab6bf60e"),
"username" : "elizabeth54",
"name" : "elizabeth",
"role" : [
"user",
"specialuser"
],
"age" : 54,
"scores" : [
10,
10,
10
],
"_class" : "com.project.ecommercemongo.model.User",
"max" : "Hello elizabeth. Your max score is 10."
},
{
"_id" : ObjectId("62c5f1f139b3d635ab6bf60f"),
"username" : "frank50",
"name" : "frank",
"role" : [
"user"
],
"age" : 50,
"scores" : [
10,
10,
10
],
"_class" : "com.project.ecommercemongo.model.User",
"max" : "Hello frank. Your max score is 10."
},
{
"_id" : ObjectId("62c5f27a39b3d635ab6bf610"),
"username" : "john26",
"name" : "john",
"role" : [
"user"
],
"age" : 26,
"scores" : [
8,
8,
10
],
"_class" : "com.project.ecommercemongo.model.User",
"max" : "Hello john. Your max score is 10."
}
]
}
This is the result I get
[
{
"_id": null,
"username": null,
"name": null,
"role": null,
"age": null,
"scores": null
}
]
this is the method that call facet addFields and function,but it don't add the field "max" and returns null
public List<User> aggregation1() {
List<Object> bl = new LinkedList<Object>();
bl.add("$name");
bl.add("$scores");
ScriptOperators.Function function= Function.function("function(name, scores) {let max = Math.max(...scores); return `Hello ${name}. Your max score is ${max}`}").args(bl).lang("js");//`Hello ${name}. Your max score is ${max}.`}");
System.out.println(function.toDocument());
FacetOperation facetOperation1 = Aggregation.facet(Aggregation.match(Criteria.where("age").gte(18)),Aggregation.addFields().addFieldWithValue("max", function).build()).as("aggregation");
Aggregation agg = Aggregation.newAggregation(facetOperation1);
return mongoTemplate.aggregate(agg, "utenti",User.class).getMappedResults();
}
And this is the User class
#Document(collection="utenti")
//#Sharded(shardKey = { "country", "userId" })
public class User {
#Id
private String _id;
private String username;
private String name;
private String[] role;
private Integer age;
private Integer[] scores;
public User() {
super();
}
public String get_id() {
return _id;
}
public void set_id(String _id) {
this._id = _id;
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String[] getRole() {
return role;
}
public void setRole(String[] role) {
this.role = role;
}
public Integer[] getScores() {
return scores;
}
public void setScores(Integer[] scores) {
this.scores = scores;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
}
how can i put all the users into the facet and add a field ? Thank you
Update
Now it work, I had to map the output with the aggregation fields
public List<MaxFacet> aggregation1() {
List<Object> bl = new LinkedList<Object>();
bl.add("$name");
bl.add("$scores");
ScriptOperators.Function function = Function.function(
"function(name, scores) {let max = Math.max(...scores); return `Hello ${name}. Your max score is ${max}.`}")
.args(bl).lang("js");// `Hello ${name}. Your max score is ${max}.`}");
System.out.println(function.toDocument());
FacetOperation facetOperation = Aggregation.facet().and(match(Criteria.where("age").gte(18)),
Aggregation.addFields().addFieldWithValue("max", function).build()).as("aggregation");
Aggregation agg = Aggregation.newAggregation(facetOperation);
return mongoOperations.aggregate(agg, mongoTemplate.getCollectionName(User.class), MaxFacet.class)
.getMappedResults();
}
public class MaxFacet {
private List<UserOut> aggregation;
public List<UserOut> getAggregation() {
return aggregation;
}
public void setAggregation(List<UserOut> facet1) {
this.aggregation = facet1;
}
public MaxFacet() {
// TODO Auto-generated constructor stub
}
}
#JsonPropertyOrder({"id","username","name","age","scores","max"})
public class UserOut {
#JsonProperty(value="id")
private String id;
private String username;
private String name;
private String []scores;
private String max;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String[] getScores() {
return scores;
}
public void setScores(String[] scores) {
this.scores = scores;
}
public String getMax() {
return max;
}
public void setMax(String value) {
this.max= value;
}
}
[
{
"aggregation": [
{
"id": "62c5f16b39b3d635ab6bf60d",
"username": "james34",
"name": "james",
"scores": [
"10",
"9",
"10"
],
"max": "Hello james. Your max score is 10."
},
{
"id": "62c5f1b839b3d635ab6bf60e",
"username": "elizabeth54",
"name": "elizabeth",
"scores": [
"10",
"10",
"10"
],
"max": "Hello elizabeth. Your max score is 10."
},
{
"id": "62c5f1f139b3d635ab6bf60f",
"username": "frank50",
"name": "frank",
"scores": [
"10",
"10",
"10"
],
"max": "Hello frank. Your max score is 10."
},
{
"id": "62c5f27a39b3d635ab6bf610",
"username": "john26",
"name": "john",
"scores": [
"8",
"8",
"10"
],
"max": "Hello john. Your max score is 10."
}
]
}
]
Alternative soluition: You can simplify your code this way
1- Use inheritance mecanism for the UserOut class
public class UserOut extends User {
private String max;
public String getMax() {
return max;
}
public void setMax(String value) {
this.max= value;
}
}
2- For the complex pipelines, the custom AggregationOperation implementation is friendlier than Spring Data builder.
AggregationOperation facetOperation = ao ->
new Document("$facet",
new Document("aggregation", Arrays.asList(
new Document("$match", new Document("age", new Document("$gt", 18))),
new Document("$addFields",
new Document("max",
new Document("$function",
new Document("body", "function(name, scores) {"+
" let max = Math.max(...scores);"+
" return `Hello ${name}. Your max score is ${max}.`"+
"}")
.append("args", Arrays.asList("$name", "$scores"))
.append("lang", "js")
)
)
)
))
);
Aggregation agg = Aggregation.newAggregation(facetOperation);
return mongoOperations
.aggregate(agg, mongoTemplate.getCollectionName(User.class), MaxFacet.class)
.getMappedResults();
Alternatively, just parse your shell JSON query this way (triple quotes starting from Java 13):
Document.parse("""
{
"$facet":
{
"aggregation": [
{
"$match": { "age": { "$gt": 18 } }
},
{
"$addFields":{
"max": {
"$function": {
"body": function(name, scores) {
let max = Math.max(...scores);
return `Hello ${name}. Your max score is ${max}.`
},
"args": [ "$name", "$scores"],
"lang": "js"
}
}
}
}
]
}
}
""")
I'd like to merge these two collections, and it has "name" variable as the common field.
// product collections
db.products.insertOne( { name : "iPhone 5", Price : 600, company : "Apple"})
db.products.insertOne( { name : "Galaxy Note9", Price : 900, company : "samsung"})
db.products.insertOne( { name : "LG V40 ThinQ", Price : 800, company : "LG"})
db.products.insertOne( { name : "Watch", Price : 400, company : "Apple"})
db.products.insertOne( { name : "iPhone 7", Price : 900, company : "Apple"})
// product_review Collection
db.product_review.insertOne( { name : "Watch", comment: " Great Product" })
db.product_review.insertOne( { name : "Watch", comment: " AMAZING" })
db.product_review.insertOne( { name : "iPhone 7", comment: " Not bad" })
db.product_review.insertOne( { name : "iPhone 7", comment: " Expeinsive" })
db.product_review.insertOne( { name : "Galaxy Note9", comment: " Great Product" })
db.product_review.insertOne( { name : "Galaxy Note9", comment: " Great Product" })
// end of product_review collection.
The output that I am looking for is as the following. One Name and one or many comments without repeating the name with each comment. It should be like this.
{
name: "iPhone 7",
Price: 900,
company: "Samsung",
comments :[
{"comment": "Not bad"},
{"comment": " Expeinsive"}
]
}
My current Code does not give me the comment, It gives me the name and price, and empty comment.
I found this code and tried to adopt it, but it can be work as I want
// map for products collection
var mapproducts = function() {
var output = {
name: this.name,
Price :this.Price,
company:this.company,
comment:0
}
if (this.Price == 0) {
output.name = this.name;
}
emit(this.name, output);
};
// map for product_review collection
var map_product_review = function() {
emit(this.name, {
name: this.name,
comment:this.comment,
Price:0,
company: 0
});
};
// Reduce function.
// I believe that the "IF condition" has to be done, is specific ways
var r = function(key, values) {
var outs = { name: 1,comment: 1, Price: 0,company:0};
values.forEach(function(v){
outs.name = v.name;
if (v.comment == 0) {
outs.Price = v.Price;
outs.company = v.company;
}
});
return outs;
};
db.products.mapReduce(mapproducts, r, {out: {reduce: ‘Result’}})
db.product_review.mapReduce(map_product_review, r, {out: {reduce: 'Result'}})
db.Result.find().pretty()
Are nested DTOs in JPQL queries not allowed in Spring Data ? :
#Query("SELECT new test.customresult.CategoryCounter(c.name, "
+ "new test.customresult.CarResult(COUNT(e.category), e.category)"
+ "FROM error e JOIN e.car c "
+ "GROUP BY c.name,e.category")
List<CategoryCounter>countErrors();
Because, I get the following error message when trying to use the above mentioned JPQL query :
Caused by: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: , near line 1,
I want to return such a JSON file from the JPQL query :
[{
"Car 1":
[{
"count": 1,
"category": "A"
}, {
"count": 2,
"category": "B"
}, {
"count": 0,
"category": "C"
}, {
"count": 0,
"category": "D"
}
]
}, {
"Car 2":
[{
"count": 0,
"category": "A"
}, {
"count": 0,
"category": "B"
}, {
"count": 4,
"category": "C"
}, {
"count": 5,
"category": "D"
}
]
}
]
There is a car table and an error table which contains the categories and a foreign key to the car table.
I wanted to use the nested DTO to represent the desired output :
The "wrapper" DTO
public class CategoryCounter {
private String name;
private CarResult carResult ;
public CategoryCounter (String name, CarResult carResult ) {
this.name= name;
this.carResult = carResult ;
}
public CarResult getCarResult () {
return carResult ;
}
public void setCarResult(CarResult carResult ) {
this.carResult = carResult ;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name= name;
}
}
and the "nested" DTO :
public class CarResult {
private Long count;
private String category;
public CarResult (Long count, String category) {
this.count = count;
this.category= category;
}
public Long getCount() {
return count;
}
public void setCount(Long count) {
this.count = count;
}
public String getCategory() {
return category;
}
public void setCategory(String category) {
this.category= category;
}
}
I'm not sure if this could be the problem... but you have your FROM clause starting immediately after the ) it says:
+ "new test.customresult.CarResult(COUNT(e.category), e.category)"
+ "FROM error e JOIN e.car c "
that is the same to: e.category)FROM so put a space in that part and try again...
+ "new test.customresult.CarResult(COUNT(e.category), e.category)"
+ " FROM error e JOIN e.car c "
the thing is that the error (Caused by: org.hibernate.hql.internal.ast.QuerySyntaxException) is a syntax error
I facing a situation where I need to query the result from a collection in MongoDB. I want to aggregate my collection based on one field using $group. But I want those remaining fields in my result, but without applying any aggregate functions on these field like ($first, $sum etc). These remaining fields should be in an Array.
Example: My collection:
-------------------------------------------------------------------------
| name | age | sex | province | city| area | address |
-------------------------------------------------------------------------
| A | 22 | m | Manglr | p1 | c1 | a1 |
| A | 22 | m | Kolkt | p2 | c2 | a2 |
| B | 24 | m | Mumb | p3 | c3 | a3 |
| B | 24 | m | Koch | p4 | c4 | a4 |
| B | 24 | m | Hydrbd | p5 | c5 | a5 |
-------------------------------------------------------------------------
Result I want: ($group by 'name' field only)
[
{
"name" : “A”,
"province" : [“Manglr", ‘Kolkt’]
"city" : [“p1”, ‘p2’],
"area" : [“c1”, ‘c2’],
"address" : [“a1”, ‘a2’],
},
{
"name" : “B”,
"province" : [“Mumb", ‘Koch’, 'Hydrbd']
"city" : [“p3”, ‘p4’,”p5”],
"area" : [“c3”, ‘c4’,”c5”],
"address" : [“a3”, ‘a4’,’a5’],
}
]
Please anyone help me to create a Mongo Query or Java code
You can try something like this. Group and push the other fields as needed.
aggregate([{
"$group": {
"_id": "$name",
"province": {
$push: {
"key": "$province"
}
},
"city": {
$push: {
"key": "$city"
}
},
"area": {
$push: {
"key": "$area"
}
},
"address": {
$push: {
"key": "$address"
}
}
}
}, {
"$project": {
"_id": 0,
"name": "$_id",
"province": "$province.key",
"city": "$city.key",
"area": "$area.key",
"address": "$address.key"
}
}])
Sample Output:
{ "province" : [ "Manglr", "kokat" ], "city" : [ "p1", "p2" ], "area" : [ "c1", "c2" ], "address" : [ "a1", "a2" ], "name" : "A" }
{ "province" : [ "Mumb", "Koch" ], "city" : [ "p3", "p4" ], "area" : [ "c3", "c4" ], "address" : [ "a3", "a4" ], "name" : "B" }
I do not know about MongoDB way but following is what you can do Java.
package com.grs.stackOverFlow.pack05;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.stream.Collectors;
class UserAccumulated{
private String name;
private List<String> city;
private List<Integer> age;
private List<Character> sex;
public UserAccumulated(){
city=new ArrayList<>();
age=new ArrayList<>();
sex=new ArrayList<>();
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public List<String> getCity() {
return city;
}
public void setCity(List<String> city) {
this.city = city;
}
public List<Integer> getAge() {
return age;
}
public void setAge(List<Integer> age) {
this.age = age;
}
public List<Character> getSex() {
return sex;
}
public void setSex(List<Character> sex) {
this.sex = sex;
}
public void addAge(Integer age2) {
age.add(age2);
}
public void addCity(String city2) {
city.add(city2);
}
public void addSex(Character sex2) {
sex.add(sex2);
}
public String toString(){
return String.format("{name:%s,cities : %s, sex: %s, age: %s}", name,city,sex,age);
}
}
public class User {
private String name,city;
private Integer age;
private Character sex;
public User(String name, String city, Integer age, Character sex) {
super();
this.name = name;
this.city = city;
this.age = age;
this.sex = sex;
}
public static void main(String...args){
//create a sample list ..you have to replace with code to retrieve data from mongo db
List<User> rows = Arrays.asList(new User("A", "Manglr", 22, 'm'),
new User("A", "Manglr", 22, 'm'),
new User("B", "addad", 22, 'm'),
new User("C", "addsadad", 22, 'm'),
new User("C", "sadd", 21, 'm'));
//aggregating
List<UserAccumulated> result=new ArrayList<>();
//parallestream if many records else use stream
Map<String, List<User>> map = rows.parallelStream().collect(Collectors.groupingBy(User::getName));
for(Entry<String, List<User>> entry: map.entrySet()){
UserAccumulated userA=new UserAccumulated();
userA.setName(entry.getKey());
for(User user : entry.getValue()){
userA.addAge(user.getAge());
userA.addCity(user.getCity());
userA.addSex(user.getSex());
}
result.add(userA);
}
for(UserAccumulated a: result)
System.out.println(a);
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getCity() {
return city;
}
public void setCity(String city) {
this.city = city;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
public Character getSex() {
return sex;
}
public void setSex(Character sex) {
this.sex = sex;
}
}
Output looks like this:
{ name: A,cities : [Manglr, Manglr], sex: [m, m], age: [22, 22]} {name:B,cities : [addad], sex: [m], age: [22]} {name:C,cities : [addsadad, sadd], sex: [m, m], age: [22, 21]}
I did not take all your columns to keep it simple. I am not sure how quickly it runs with your actual data volume. But if performance issue let me know.