I'm trying to figure out how to persist a semi-structured document to MongoDB, using Spring Mongo. Here's an example of what I'm trying to accomplish:
{
"name": "Test",
"admin": false,
"unstructured_field_one": "Some arbitrary data"
}
I know how to do this with fully unstructured data as a field in a parent document, where I can just use something like
//...
private Object someRandomObject;
But how can I accomplish a semi-structured document (at parent level), where I have, as in the example, name and admin as required fields, and whatever else comes in along with the request gets added automagically?
You can do it without any pojo, just with a Json Parser(Jackson) and MongoTemplate. Since MongoTemplate can save any DbObject, You need to convert your json to a DBObject.
Something like this will do
ObjectMapper mapper = new ObjectMapper();
TypeReference<Map<String,Object>> typeRef
= new TypeReference<Map<String,Object>>() {};
Map<String,Object> map = mapper.readValue(json, typeRef);
DBObject dbObject = new BasicDBObject(map);
mongoTemplate.getCollection("blahblah").save(dbObject);
Related
I need to migrate a Spring Boot project to Quarkus. The project has been using Spring Data Mongodb for all the queries. Now I find it difficult to migrate complex queries.
One example is
public List<MyData> findInProgressData(MyConditions conditions) {
Query mongoQuery = new Query();
mongoQuery.addCriteria(Criteria.where("status").is(IN_PROGRESS));
mongoQuery.addCriteria(Criteria.where("location").is(conditions.getLocationCode()));
if (StringUtils.isNotBlank(conditions.getType())) {
mongoQuery.addCriteria(Criteria.where("type").is(conditions.getType()));
}
if (StringUtils.isNotBlank(conditions.getUserId())) {
mongoQuery.addCriteria(Criteria.where("userId").is(conditions.getUserId()));
}
mongoQuery.with(Sort.by(Sort.Direction.ASC, "createdAt"));
return mongoTemplate.find(mongoQuery, MyData.class);
}
How can I implement the conditional query with Quarkus?
You can probably try Panache with Mongodb.
You can pass Document into .find() method, In this object you can specify any criteria.
final Document document = new Document();
document.put("status","IN_PROGRESS");
...
result = MyData.find(document); // or MyDataRepository
But you'll need to adapt some of the code to Panache, which can be done either via extending PanacheEntity or PanacheEntityBase
Before, using spring data mongo you could do something like mongoClient.getDB(db_name).getCollection(collection_name).isCapped(). But now the getDB is deprecated, you could still use it but there must be some other way to do it.
I tried doing mongoClient.getDatabase(db_name).getCollection(collection_name).some_function() but there is no similar function like isCapped() now.
You can achieve to find if a collection is capped by doing the following.
MongoTemplate mongoTemplate = new MongoTemplate(mongoClient(), getDatabaseName());
Document obj = new Document();
obj.append("collStats", "yourCollection");
Document result = mongoTemplate.executeCommand(obj);
result.getBoolean("capped")
I think this issue probably has to do with my Mongo Document Koltin Data class, but for our business case we need to allow the user to add on any JSON fields to describe their RF data set.
Extending the BasicDBObject was the best way I have found.
The mono being returned when I save a SigMfMetaDocument does not contain the _id field.
I cannot figure out why the save method does not return a Mono wrapping a SigMfDocument with and _id
If there is a better way to create a Type for ReactiveMongoRepository that can dynamically accept any fields I am all ears.
#Document(collection = "sigmfmeta")
class SigMfMetaDocument : BasicDBObject {
#Id
#JsonProperty("id")
val id: String? = UUID.randomUUID().toString()
constructor(map: Map<String, Any>) : super(map)
constructor() : super()
constructor(key: String, value: Object): super()
}
#Repository
interface SigMfMetaRepository : ReactiveMongoRepository<SigMfMetaDocument, String>
So I found a way to solve this for my use case. I was originally assuming the description in the documentation for the save method would apply
(Saves a given entity. Use the returned instance for further operations as the save operation might have changed the entity instance completely).
My thought Mongo auto inserting the _id value would apply to this description.
I changed my model to:
#Document(collection = "sigmfmeta")
class SigMfMetaDocument : BasicBSONObject {
constructor(map: Map<String, Any>) : super(map) {
val id = ObjectId()
this.put("_id", id)
}
constructor() : super()
}
This way I have the _id value after saving for some business logic. Again I defined my Model this way because the metadata file we are accepting needs to allow a client to add any fields they wish to describe a binary file of RF measurement data.
I have a MongoDB document that I need to update using Casbah for scala. My mongoDB document looks like
{"_id": ObjectId("58d86364fbb1bb2224cab56a"),
"record_interval":
[
{
"record_time": ISODate("2017-01-26T09:22:15.000Z"),
"insert_time": ISODate("2017-03-26T12:57:08.610Z"),
"reading1": 50.0,
"reading2": 627.0
}
],
"record_id": "1234",
"record_hour": ISODate("2017-01-26T09:00:00.000Z")
}
I inserted the above document using df.write methodology, so I was able to specify the schema with datatypes when I created the dataframe and was able to successfully insert the document with the the specified datatypes.
Now, I need to add an object inside the record interval array. I have a JSON string that I parsed as a DBObject
val DBobject: DBObject = JSON.parse(Json_String).asInstanceOf[DBObject]
The DBobject that looks like below
{
"vib_temp": "55.0",
"vib_voltage": "647.0",
"message_time": "2017-01-26 03:48:52.000000",
"etl_date_time": "2017-03-26 06:57:09.302000"
}
I added this DBObject into the record_interval array of the aforementioned document using the below code.
collection.update(MongoDBObject("_id" -> new ObjectId("58d86364fbb1bb2224cab56a"))
,$push("record_interval" -> new MongoDBObject(DBobject)))
I am able to update the desired document, but the datatypes of the elements record_time, insert_time, reading1 and reading2 are all strings. Whereas I would like to insert the object with appropriate datatypes. How do I specify the datatypes while updating the document? Thanks in advance
I found that the DBObject that I am trying to insert should have the values with the desired data type.
Following worked for me.
val Current_datetime = new DateTime()
val message_time = new DateTime()
collection.update(MongoDBObject("_id" -> new ObjectId("58d86364fbb1bb2224cab56a"))
,$push("record_interval" -> new MongoDBObject(
,("vib_temp"->55.0)
,("vib_voltage"->647.0)
,("message_time"->message_time)
,("etl_date_time"-> Current_datetime))))
I need to iterate through all of the collections in my MongoDB database and get the time when each of the collections was created (I understand that I could get the timestamp of each object in the collection, but I would rather not go that route if a simpler/faster method exists).
This should give you an idea of what I'm trying to do:
MongoDatabase _database;
// code elided
var result = _database.GetAllCollectionNames().Select(collectionName =>
{
_database.GetCollection( collectionName ) //.{GetCreatedDate())
});
As far as I know, MongoDB doesn't keep track of collection creation dates. However, it's really easy to do this yourself. Add a simple method, something like this, and use it whenever you create a new collection:
public static void CreateCollectionWithMetadata(string collectionName)
{
var result = _db.CreateCollection(collectionName);
if (result.Ok)
{
var collectionMetadata = _db.GetCollection("collectionMetadata");
collectionMetadata.Insert(new { Id = collectionName, Created = DateTime.Now });
}
}
Then whenever you need the information just query the collectionMetadata collection. Or, if you want to use an extension method like in your example, do something like this:
public static DateTime GetCreatedDate(this MongoCollection collection)
{
var collectionMetadata = _db.GetCollection("collectionMetadata");
var metadata = collectionMetadata.FindOneById(collection.Name);
var created = metadata["Created"].AsDateTime;
return created;
}
The "creation date" is not part of the collection's metadata. A collection does not "know" when it was created. Some indexes have an ObjectId() which implies a timestamp, but this is not consistent and not reliable.
Therefore, I don't believe this can be done.
Like Mr. Gates VP say, there is no way using the metadata... but you can get the oldest document in the collection and get it from the _id.
Moreover, you can insert an "empty" document in the collection for that purpose without recurring to maintain another collection.
And it's very easy get the oldest document:
old = db.collection.find({}, {_id}).sort({_id: 1}).limit(1)
dat = old._id.getTimestamp()
By default, all collection has an index over _id field, making the find efficient.
(I using MongoDb 3.6)
Seems like it's some necroposting but anyway: I tried to find an answer and got it:
Checked it in Mongo shell, don't know how to use in C#:
// db.payload_metadata.find().limit(1)
ObjectId("60379be2bec7a3c17e6b662b").getTimestamp()
ISODate("2021-02-25T12:45:22Z")