I am both new to scala and cashbah. I am trying to
update a document if exists (by _id) and create if doesn't exist.
while updating, update some key values
while updating, update some keys which values are sets, include some data to those sets.
To achive this, I've written this:
DBObject = MongoDBObject("_id" -> uri.toString) ++
$addToSet("appearsOn" -> sourceToAppend) ++
$addToSet("hasElements" -> elementsToAppend) ++
$addToSet("hasTriples" -> triplesToAppend) ++
MongoDBObject("uDate" -> new DateTime)
/* Find and replace here! */
OntologyDocument.dao.collection.findAndModify(
query = MongoDBObject({"_id" -> uri.toString}),
update = update,
upsert = true,
fields = null,
sort = null,
remove = false,
returnNew = true
)
Documents looked by _id, some new items added to appearsOn hasElements hasTriples and uDate is updated.
sourceToAppend elementsToAppend and triplesToAppend are List[String]
When I run this, I got this error:
java.lang.IllegalArgumentException: fields stored in the db can't start with '$' (Bad Key: '$addToSet')
at com.mongodb.DBCollection.validateKey(DBCollection.java:1444) ~[mongo-java-driver-2.11.1.jar:na]
I didn't get it. What is wrong with this query? $addToSet isn't a field, why casbah thinks it is a field? What am I doing wrong here?
The reason its failing is because the update query is invalid (it wont work in the js shell).
$set is implicit for values in the update document, but you can't mix it with other update operators eg $addToSet. If you want to mix $set with other set operators then you can if you are explicit:
val update = $set("uDate" -> new DateTime) ++
$addToSet("appearsOn" -> sourceToAppend,
"hasElements" -> elementsToAppend,
"hasTriples" -> triplesToAppend)
You can't $set "_id" but as thats in the query and its an upsert - it will merge so don't include it in the update statement - otherwise it will error.
Finally, #AsyaKamsky is right if you dont need the returned document - use an update its also atomic.
Related
I want to update FieldA in someFields while someFields is null in MongoDB
{
"someFileds : {
"FieldA" : "ABS"
}
}
someFields may have more fields as well but Just want to ensure that other fields won't be overridden.
Sometime SomeFields won't be having any fields so want to
I am using below code for set the the value however value for someFields is null in MongoDB.
BasicDBObject dbObject = new BasicDBObject();
dbObject.put("SomeFields.FieldA", "TEST");
Could you please suggest how can handle the null in this scenario.
Use this Mongo update query.
db.collection.update({}, {$set:{"someFields": {"FieldA": "Test"}}});
It will create FieldA in someFields when it is null otherwise will only update FieldA.
In your case kindly pass object all time in 2nd parameter of put.
dbObject.put("SomeFields", {FieldA:"TEST"});
I am trying to get an aggregate in ReactiveMongo 0.12 and Play Framework 2.6 (using JSON collections - not BSON) by filtering dates from a collection called "visitors". A typical document may look like this:
{ "_id": ObjectID("59c33152ca2abb344c575152"), "placeId": ObjectID("59c33152ca2abb344c575152"), "date": ISODate("2017-03-26T00:00:00Z"), "visitors": 1200 }
So from here I want to aggregate this data to get various visitor totals, averages, etc, grouping by placeId (which identifies the place in another collection) and filtering by dates after 15-05-2016.
I've based this on this similar question - without the match it works but with it - it does not. There isn't an error but it just doesn't work:
def getVisitorAggregate(col: JSONCollection) = {
import col.BatchCommands.AggregationFramework.{Group, Match, SumField, AvgField, MinField, MaxField}
val format = new java.text.SimpleDateFormat("dd-MM-YYYY")
val myDate = "15-05-2016"
val parseDate: Date = format.parse(myDate)
val longDate: Long = parseDate.getTime
col.aggregate(
Group(JsString("$placeId"))(
"totalVisitors" -> SumField("visitors"),
"avgVisitors" -> AvgField("visitors"),
"minVisitors" -> MinField("visitors"),
"maxVisitors" -> MaxField("visitors")
),
List(Match(Json.obj("date" -> Json.obj("$gte" -> JsNumber(longDate)))))
)
.map(_.head[VisitorAggregate])
}
I have looked and tested for many hours online and I cannot find the correct syntax but this will be simple for someone who knows I'm sure. Thanks
ISODate is a mongodb type, and Model.aggregate does not cast the arguments, so "date" -> Json.obj("$gte" -> JsNumber(longDate)) is wrong.
You need to use a type that will be converted to the ISODate, I am pretty sure it is not JsNumber.
It is a BSONDateTime type would you use BSON, but you do not.
According to documentation it must be a
JsObject with a $date JsNumber field with the timestamp (milliseconds)
as value
So solution can be (I did not verify):
Match(Json.obj("date" -> Json.obj("$gte" -> Json.obj("$date" -> JsNumber(longDate)))))
I hate to answer my own question here but now that I have figured this out I really want to clarify to others how this is done using Aggregate. Ultimately there were two parts to this question.
1) what is the syntax of querying dates?
As #AndriyKuba mentioned and I had seen in the documentation yet not fully understood; the query is formulated like this:
Json.obj("date" -> Json.obj("$gte" -> Json.obj("$date" -> JsNumber(longDate))))
2) how do I match a query within an Aggregate?
This is more of a question of the order of the query. I was originally trying to use match after grouping and aggregating the data - which is (obviously) only going to filter the data after. As I wanted to first get a date range and then aggregate that data I had to match first - this also meant that some of the syntax had to change accordingly:
def getVisitorAggregate(col: JSONCollection) = {
import col.BatchCommands.AggregationFramework.{Group, Match, SumField, AvgField, MinField, MaxField}
val format = new java.text.SimpleDateFormat("dd-MM-YYYY")
val myDate = "15-05-2016"
val parseDate: Date = format.parse(myDate)
val longDate: Long = parseDate.getTime
col.aggregate(
Match(Json.obj("date" -> Json.obj("$gte" -> Json.obj("$date" -> JsNumber(longDate))))),
List(Group(JsString("$rstId"))(
"totalVisitors" -> SumField("visitors"),
"avgVisitors" -> AvgField("visitors"),
"minVisitors" -> MinField("visitors"),
"maxVisitors" -> MaxField("visitors")
))
)
.map(_.head[VisitorAggregate])
}
Really frustrating that there isn't more documentation out there on using the Play Framework with ReactiveMongo as there are a lot of instances of trying to fathom syntax and logic.
That is meant to be read as a dual upsert operation, upsert the document then the array element.
So MongoDB is a denormalized store for me (we're event sourced) and one of the things I'm trying to deal with is the concurrent nature of that. The problem is this:
Events can come in out of order, so each update to the database need to be an upsert.
I need to be able to not only upsert the parent document but an element in an array property of that document.
For example:
If the document doesn't exist, create it. All events in this stream have the document's ID but only part of the information depending on the event.
If the document does exist, then update it. This is the easy part. The update command is just written as UpdateOneAsync and as an upsert.
If the event is actually to update a list, then that list element needs to be upserted. So if the document doesn't exist, it needs to be created and the list item will be upserted (resulting in an insert); if the document does exist, then we need to find the element and update it as an upsert, so if the element exists then it is updated otherwise it is inserted.
If at all possible, having it execute as a single atomic operation would be ideal, but if it can only be done in multiple steps, then so be it. I'm getting a number of mixed examples on the net due to the large change in the 2.x driver. Not sure what I'm looking for beyond the UpdateOneAsync. Currently using 2.4.x. Explained examples would be appreciated. TIA
Note:
Reiterating that this is a question regarding the MongoDB C# driver 2.4.x
Took some tinkering, but I got it.
var notificationData = new NotificationData
{
ReferenceId = e.ReferenceId,
NotificationId = e.NotificationId,
DeliveredDateUtc = e.SentDate.DateTime
};
var matchDocument = Builders<SurveyData>.Filter.Eq(s => s.SurveyId, e.EntityId);
// first upsert the document to make sure that you have a collection to write to
var surveyUpsert = new UpdateOneModel<SurveyData>(
matchDocument,
Builders<SurveyData>.Update
.SetOnInsert(f => f.SurveyId, e.EntityId)
.SetOnInsert(f => f.Notifications, new List<NotificationData>())){ IsUpsert = true};
// then push a new element if none of the existing elements match
var noMatchReferenceId = Builders<SurveyData>.Filter
.Not(Builders<SurveyData>.Filter.ElemMatch(s => s.Notifications, n => n.ReferenceId.Equals(e.ReferenceId)));
var insertNewNotification = new UpdateOneModel<SurveyData>(
matchDocument & noMatchReferenceId,
Builders<SurveyData>.Update
.Push(s => s.Notifications, notificationData));
// then update the element that does match the reference ID (if any)
var matchReferenceId = Builders<SurveyData>.Filter
.ElemMatch(s => s.Notifications, Builders<NotificationData>.Filter.Eq(n => n.ReferenceId, notificationData.ReferenceId));
var updateExistingNotification = new UpdateOneModel<SurveyData>(
matchDocument & matchReferenceId,
Builders<SurveyData>.Update
// apparently the mongo C# driver will convert any negative index into an index symbol ('$')
.Set(s => s.Notifications[-1].NotificationId, e.NotificationId)
.Set(s => s.Notifications[-1].DeliveredDateUtc, notificationData.DeliveredDateUtc));
// execute these as a batch and in order
var result = await _surveyRepository.DatabaseCollection
.BulkWriteAsync(
new []{ surveyUpsert, insertNewNotification, updateExistingNotification },
new BulkWriteOptions { IsOrdered = true })
.ConfigureAwait(false);
The post linked as being a dupe was absolutely helpful, but it was not the answer. There were a few things that needed to be discovered.
The 'second statement' in the linked example didn't work
correctly, at least when translated literally. To get it to work, I had to match on the
element and then invert the logic by wrapping it in the Not() filter.
In order to use 'this index' on the match, you have to use a
negative index on the array. As it turns out, the C# driver will
convert any negative index to the '$' character when the query is
rendered.
In order to ensure they are run in order, you must include bulk write
options with IsOrdered set to true.
I have a MongoDB document that I need to update using Casbah for scala. My mongoDB document looks like
{"_id": ObjectId("58d86364fbb1bb2224cab56a"),
"record_interval":
[
{
"record_time": ISODate("2017-01-26T09:22:15.000Z"),
"insert_time": ISODate("2017-03-26T12:57:08.610Z"),
"reading1": 50.0,
"reading2": 627.0
}
],
"record_id": "1234",
"record_hour": ISODate("2017-01-26T09:00:00.000Z")
}
I inserted the above document using df.write methodology, so I was able to specify the schema with datatypes when I created the dataframe and was able to successfully insert the document with the the specified datatypes.
Now, I need to add an object inside the record interval array. I have a JSON string that I parsed as a DBObject
val DBobject: DBObject = JSON.parse(Json_String).asInstanceOf[DBObject]
The DBobject that looks like below
{
"vib_temp": "55.0",
"vib_voltage": "647.0",
"message_time": "2017-01-26 03:48:52.000000",
"etl_date_time": "2017-03-26 06:57:09.302000"
}
I added this DBObject into the record_interval array of the aforementioned document using the below code.
collection.update(MongoDBObject("_id" -> new ObjectId("58d86364fbb1bb2224cab56a"))
,$push("record_interval" -> new MongoDBObject(DBobject)))
I am able to update the desired document, but the datatypes of the elements record_time, insert_time, reading1 and reading2 are all strings. Whereas I would like to insert the object with appropriate datatypes. How do I specify the datatypes while updating the document? Thanks in advance
I found that the DBObject that I am trying to insert should have the values with the desired data type.
Following worked for me.
val Current_datetime = new DateTime()
val message_time = new DateTime()
collection.update(MongoDBObject("_id" -> new ObjectId("58d86364fbb1bb2224cab56a"))
,$push("record_interval" -> new MongoDBObject(
,("vib_temp"->55.0)
,("vib_voltage"->647.0)
,("message_time"->message_time)
,("etl_date_time"-> Current_datetime))))
How to make sure that specific fields can be inserted upon creation, but can optionally be excluded when updating the object.
I'm essentially looking for something like the following:
mongoOperations.save(theObject, <fields to ignore>)
From what I see, recently introduced #ReadOnlyProperty will ignore the property both for inserts and updates.
I was able to get the desired behavior by implementing my Custom MongoTemplate and overriding its doUpdate method as follows:
#Override
protected WriteResult doUpdate(String collectionName, Query query,
Update originalUpdate, Class<?> entityClass, boolean upsert, boolean multi) {
Update updateViaSet = new Update();
DBObject dbObject = originalUpdate.getUpdateObject();
Update filteredUpdate = Update.fromDBObject(dbObject, "<fields to ignore>");
for(String key : filteredUpdate.getUpdateObject().keySet()){
Object val = filteredUpdate.getUpdateObject().get(key);
System.out.println(key + "::" + val);
updateViaSet.set(key, filteredUpdate.getUpdateObject().get(key));
}
return super
.doUpdate(collectionName, query, updateViaSet, entityClass, upsert, multi);
}
But the issue is that now it will use Mongo $set form of updates for everything, not just for specific cases.
Please advise if there is any simpler (and correct) way to achieve this.
While creating an update object, use $setOnInsert instead of $set. It is available in spring mongo as well.
If an update operation with upsert: true results in an insert of a document, then $setOnInsert assigns the specified values to the fields in the document. If the update operation does not result in an insert, $setOnInsert does nothing.