We noticed an issue after an upgrade to grails 3 that we were saving mongoDB documents with both _id and id. (example document below)
How do we stop the saving of id? This happens for every collection the application creates and updates documents for.
{
"_id" : ObjectId("5b0ed1b710b3641a98aaee63"),
"value" : "testing",
"type" : "testingCreate",
"updateDate" : ISODate("2018-05-30T16:30:39.987Z"),
"updateUser" : "TSTUSR",
"id" : ObjectId("5b0ed1b710b3641a98aaee63")
}
The save is being called from the following
def test = new AppParam(type: "testingCreate",
updateUser: "TSTUSR",
updateDate: new Date(),
value: "testing")
test.save(failOnError:true, flush:true)
for the appParam domain of
class AppParam {
ObjectId id
String type
String value
String updateUser
Date updateDate
static mapWith = "mongo"
static mapping = {
version false
writeConcern WriteConcern.ACKNOWLEDGED
}
static constraints = {
type size: 1..50, matches:/^[^<>]{1,50}$/, validator: { field, obj ->
if (!field.trim()) return ['typeRequired']
return true
}
value size: 1..2000, matches:/^[^<>]{1,2000}$/, validator: { field, obj ->
if (!field.trim()) return ['valueRequired']
return true
}
}
}
We are using grailsVersion 3.2.11 and gormVersion 6.1.7.RELEASE
Try the following in the mapping closure.
static mapping {
id column: '_id'
version false
writeConcern WriteConcern.ACKNOWLEDGED
}
Did a bit more research into Mike W's comment after Grails 3.X it should be defaulting the mongodb engine to codec and we were manually defaulting the mongodb.engine = "mapping".
Related
I have a class User that embeds a JsonObject to represent the user's fields. This class looks like that:
class User {
private JsonObject data;
public User(...) {
data = new JsonObject();
data.put("...", ...).put(..., ...);
}
public String getID() { return data.getString("_id"); }
// more getters, setters
// DB access methods
public static userSave(MongoClient mc, User user){
// some house keeping
mc.save("users", user.jsonObject(), ar -> {
if(ar.succeeded()) { ... } else { ... }
});
}
}
I've just spent more than half a day trying to figure out why a call to user.getID() sometimes produced the following error: ClassCastException: class io.vertx.core.json.JsonObject cannot be cast to class java.lang.CharSequence. I narrowed down to the userSave() method and more specifically to MongoClient::save() which actually produces a side effect which transforms the data._id from something like
"_id" : "5ceb8ebb9790855fad9be2fc"
into something like
"_id" : {
"$oid" : "5ceb8ebb9790855fad9be2fc"
}
This is confirmed by the vertx documentation which states that "This operation might change _id field of document parameter". This actually is also true for other write methods like inserts.
I came with two solutions and few questions about doing the save() properly while keeping the _id field up to date.
S1 One way to achieve that is to save a copy of the Json Object rather than the object itself, in other words : mc.save("users", user.jsonObject().copy(), ar -> {...});. This might be expensive on the long run.
S2 An other way is to "remember" _id and then to reinsert it into the data object in the if(ar.succeeded()) {data.put("_id", oidValue); ...} section. But as we are asynchronous, I don't think that the interval between save() and the data.put(...) is atomic ?
Q1: Solution S1 make the assumption that the ID doesn't change, i.e., the string 5ceb8ebb9790855fad9be2fc will not change. Do we have a warranty about this ?
Q2: What is the right way to implement the saveUser() properly ?
EDIT: The configuration JSON object user for the creation of the MongoClient is as follows (in case there is something wrong) :
"main_pool" : {
"pool_name" : "mongodb",
"host" : "localhost",
"port" : 27017,
"db_name" : "appdb",
"username" : "xxxxxxxxx",
"password" : "xxxxxxxxx",
"authSource" : "admin",
"maxPoolSize" : 5,
"minPoolSize" : 1,
"useObjectId" : true,
"connectTimeoutMS" : 5000,
"socketTimeoutMS" : 5000,
"serverSelectionTimeoutMS" : 5000
}
I'm using AWS appsync + DynamoDB.
The problem: I created the new field 'rating' in my 'Users' schema:
type Users {
id: ID!
first: String!
last: String!
rating: String #<----The new field
}
AppSync created all the resources and I can create new records with Mutations and that works like a charm.
mutation createUsers{
createUsers(input:{
first:"John"
last:"Smith"
rating:"B" #<---Writing new field without problem
}){
id
first
last
rating #<---Confirming that is recorded in DynamoDB
}
}
The problem is that I can't figure out how to write the resolver to make the following query work.
query{
queryUsersByRating(rating: "B"){
items{
id
username
rating
}
}
}
The result is this:
{
"data": {
"queryUsersByRating": null
}
}
The problem is clearly identified here under "Missing Resolver", but there's no clear solution.
I tried attaching the following Resolver directly in AppSync interface but is not working:
{
"version" : "2017-02-28",
"operation" : "Query",
"query" : {
"expression": "rating = :rating",
"expressionValues" : {
":rating" : $util.dynamodb.toDynamoDBJson($ctx.args.rating)
}
}
}
Any help would be appreciated, THANKS!
You don't have to write your own resolver for querying by rating, Appsync wrapped all the fields inside filter.
query{
queryUsersByRating(filter: {rating: "B"}){
items{
id
username
rating
}
}
}
This question is mostly popular but with a slight twist; I need to delete few records by when it was created, using its _id. I do not have any date, createdAt fields as I see that mongo uses its _id for the createdAt timestamp.
How do I delete a recored, say created 30 days ago, using this gist?
const date = new Date();
const daysToDeletion = 30;
const deletionDate = new Date(date.setDate(date.getDate() - daysToDeletion));
const db = <your-database>
db.messages.remove({_id : {$lt : deletionDate}});
The above returns a CastError
What would work, and as Ive said, I do not have a createdAt field:
db.messages.remove({createdAt : {$lt : deletionDate}});
Use mongo shell:
db.collection.find({"_id": { "$gt": ObjectId.fromDate(new Date('2017-10-01'))}})
Use mongoose:
Model.find({ _id: { $gt: mongoose.Types.ObjectId.createFromTime(new Date('2018-01-01')) } })
will find the needed docs.
var id = new ObjectId( Math.floor(deletionDate .getTime() / 1000).toString(16) + "0000000000000000")
db.messages.remove({createdAt : {$lt : id}});
You need to extract the timestamp from the object _id. So you could do something like:-
db.messages.find({}).forEach(function(rec) {
var recDate = rec._id.getTimestamp();
// Then here cast your deletionDate and recDate to preferred string format, e.g:-
var recDateString = recDate.getMonth()+1+'-'+recDate.getDate()+'-'+recDate.getFullYear();
var deletionDateString = deletionDate.getMonth()+1+'-'+deletionDate.getDate()+'-'+deletionDate.getFullYear();
if (deletionDateString == recDateString ){
db.messages.remove({_id:rec._id});
}
}
I'm having a lot of trouble getting query results for certain collections in Meteor.
I have set
idGeneration : 'MONGO'
in the collection definitions, and in the mongo shell these collections look like this :
the document i want, call it documentW (from CollectionA) = {
"_id" : ObjectId("7032d38d35306f4472844be1"),
"product_id" : ObjectId("4660a328bd55247e395edd23"),
"producer_id" : ObjectId("a5ad120fa9e5ce31926112a7") }
documentX (from collection "Products") = {
_id : ObjectId("4660a328bd55247e395edd23")
}
documentY (from collection "Producers") = {
_id : ObjectId("a5ad120fa9e5ce31926112a7")
}
If i run a query like this in Meteor
CollectionA.findOne({ product_id : documentX._id, producer_id : documentY._id})
I'm expecting to get my documentW back... but I get nothing.
When I run this query in the mongo shell
db.collectiona.find({ product_id : ObjectId("4660a328bd55247e395edd23"), producer_id :
ObjectId("a5ad120fa9e5ce31926112a7") })
I get my documentW back no problem.
Of course in Meteor if I call
console.log(documentX._id)
I get this
{ _str : "4660a328bd55247e395edd23" }
Anyone have any ideas what is going on here ? I have tried all kinds of things like
Meteor.Collection.ObjectID(documentX._id._str)
but the search still returns empty...
Running the latest 0.7.0.1 version of Meteor...
I don't know if this answers your question, but I can't put this code in a comment. This code is working for me, trying to follow what I believe you are trying to do:
Products = new Meteor.Collection("products", {
idGeneration: "MONGO"
});
Producers = new Meteor.Collection("producers", {
idGeneration: "MONGO"
});
CollectionA = new Meteor.Collection("a", {
idGeneration: "MONGO"
});
Products.insert({
foo: "bar"
});
Producers.insert({
fizz: "buzz"
});
var documentX = Products.findOne();
var documentY = Producers.findOne();
CollectionA.insert({
product_id: documentX._id,
producer_id: documentY._id
});
var documentW = CollectionA.findOne({
product_id: documentX._id,
producer_id: documentY._id
});
console.log(documentW); // This properly logs the newly created document
This is on 0.7.0.1. Do you see anything in your code that diverges from this?
Grails makes it easy to get a domain object by ID (handy for building a REST API).
A controller to retrieve a resource can be as simple as:
MetricController.groovy
import grails.converters.JSON
class MetricController {
def index() {
def resource = Metric.get(params.id)
render resource as JSON
}
}
When using the Grails plugin for MongoDB GORM (compile ":mongodb:1.2.0"), the default id type of Long needs to be changed to type String or ObjectId.
Metric.groovy
import org.bson.types.ObjectId
class Metric {
static mapWith = "mongo"
ObjectId id
String title
}
However, doing a .get(1) will now result in:
Error 500: Internal Server Error
URI
/bow/rest/metric/1
Class
java.lang.IllegalArgumentException
Message
invalid ObjectId [1]
I took a guess and changed the controller to use findById:
def resource = Metric.findById(new ObjectId(new Date(), params.id.toInteger()))
That fixed the error, but it fails to find the object (always returns null).
For example, using the id of "-1387348672" does not find this test object:
{ "class" : "Metric",
"id" : { "class" : "org.bson.types.ObjectId",
"inc" : -1387348672,
"machine" : 805582250,
"new" : false,
"time" : 1371329632000,
"timeSecond" : 1371329632
},
"title" : "Test"
}
The ObjectId.inc field may not even be the correct field to use for the resource ID.
So, what is the simplest way to retrieve a domain object by ID when using MongoDB?
When a domain object is persisted in MongoDB, it is stored as a document with an ObjectId as a unique 12 bytes BSON primary key. For example, if you have a domain object Product like
import org.bson.types.ObjectId
class Product {
ObjectId id
String name
static mapWith = "mongo"
}
then the persisted entity in MongoDB would look like below if you save with a name as "TestProduct".
//db.product.find() in mongodb
{
"_id" : ObjectId("51bd047c892c8bf0b3a58b21"),
"name" : "TestProduct",
"version" : 0
}
The _id becomes your primary key for that document. To get this document you need the primary key ObjectId. Speaking from a RESTful context, you atleast need the 12 bytes hexcode 51bd047c892c8bf0b3a58b21 as part of the request.
So in the above case, you can fetch that particular document by doing something like
Product.get(new ObjectId("51bd047c892c8bf0b3a58b21"))
Product.findById(new ObjectId("51bd047c892c8bf0b3a58b21"))
Have a look at the API for ObjectId which would make clear how to retrieve a document.
When you retrieve the document as JSON, then it just shows the ObjectId class with its elements.
{
"class": "com.example.Product",
"id": {
"class": "org.bson.types.ObjectId",
"inc": -1280996575,
"machine": -1993569296,
"new": false,
"time": 1371341948000,
"timeSecond": 1371341948
},
"name": "TestProduct"
}
For completeness, here's the domain with a controller to get a resource by ID string (instead of ObjectID).
Example:
Metric.get("51bf8ccc30040460f5a05579")
Domain
import org.bson.types.ObjectId
class Metric {
ObjectId id
String title
static mapWith = "mongo"
def out() {
return [
id: id as String, //convert Mongo ObjectId to 12-byte hex BSON
title: title
]
}
}
The out() method is used to render the resource showing its ID string (not its ObjectID).
Controller
import grails.converters.JSON
class MetricController {
def index() {
def resource = Metric.read(params.id)
render resource.out() as JSON
}
}
Example JSON response for /rest/metric/51bf8ccc30040460f5a05579
{ "id": "51bf8ccc30040460f5a05579", "title": "Accounts" }