To ease load on my MongoDB server, I want to cache some objects from Waterline in Redis. To achieve this, I have to serialize the object to JSON.
My question is, how can I construct my JSON back to an instance of the Waterline model, with datatype handling, member functions etc, making this transparent to the consuming code?
I have also wanted this whenever I run native MongoDB queries, giving me objects with native ObjectIDs, mismatching date types etc.
User.findOne(id, function (err, user) {
// to string and back again, could be stored in cache in the meantime
var object = JSON.parse(JSON.stringify(user));
var user = new User(object); //doesn't work
var user = User.toObject(object); // doesn't work
}
According to this issue, this should work
var user = new User._model(object);
Pay attention about the values that you insert. The new object should really fit what you had before the stringification.
What kind of cache do you want to approach?
var object = JSON.parse(JSON.stringify(user)); won't store all of the information. JSON is different from JavaScript object. Which user is JavaScript object, that is created by Sails's Waterline engine. It has methods like save, but when you turn it into JSON, the method will be gone.
Either var user = new User(object); or var user = User.toObject(object); won't work, because User is not a constructor, it just a shorthand to sails.models.user which represent the User model at /api/models/User.js.
If you want to query fast, you must store your data in your Redis, and retrieve from it later.
By that mean, you should save your user from the main database to your redis by insert it manually to your redis, then retrieve it later. But all of your main database model characteristic is will be gone, except you copy your main database model (User in this case) to your redis User model (or whatever you named it).
Related
I have created Core Data database, and it has an entity called Item.
If I want to create an instance of Item and add it to the database, I create it using Item(context: ).
How can I create an instance of Item without adding it to the Core Data database?
I tried something like this, but the app crashes, when I run this code:
extension Item {
static var sampleItem: Item {
let sampleItem = Item()
sampleItem.timestamp = Date.now
return sampleItem
}
}
Without a context (and by extension, without any connection to a persistent store or managed object model), core data can't figure out how your object should behave.
If you're not interested in persistence or relationships to any actual objects, then you can create an in-memory persistent store, using the same managed object model, and create objects in there. Those objects can also be used for unit tests and SwiftUI previews.
If you are interested in throwaway, but "real" objects, that would represent real things from the database and their relationships, then create a child main queue context with a parent of your main context. Real objects from your database will then be available, and you can do whatever you like with them. Unless you save the child context, the original database will be untouched.
You can do this if you use init(entity:insertInto:) to create the instance, because the context argument is optional. This means you have to look up the entity description first.
You get the description from a managed object context, which would be something like
let itemEntity = NSEntityDescription.entity(forEntityName: "Item",
in: container.viewContext)!
Then you can create an instance like this:
let uninsertedItem = Item(entity: personEntity, insertInto:nil)
Then you can work with uninsertedItem. Later on if you want to save it, insert it into a context:
container.viewContext.insert(uninsertedItem)
If you don't want to save it, don't do that, just let the object get deallocated.
I would really like to avoid using NSManagedObjectID as a way to connect my model structs to their CoreData objects. I mean something like this:
Say I have a Book entity in CoreData and then I have a model struct like this representing it for my model layer:
struct BookModel {
let name: String
...
let objectID: NSManagedObjectID // I need this to refer back to the entry in the database
}
I don't like this approach. It makes working with the structs tedious and, for instance, testing is annoying because I always have to generate dummy objectIds or make BookModel.objectID optional.
What I would love to have is an id property of type UUID inside the Book entity. This would be so easy to connect to structs and also allows the structs to properly exist without a database:
struct BookModel {
let name: String
...
let id: UUID
...
func object() -> Book {
// Retrieve managed object using a fetch request with a predicate.
}
}
I've noticed that you can actually have UUID properties in an entity. However, the performance difference seems to be enormous. I've created an example that tries to fetch individual objects 10000 times.
First, I fetched them using the contexts object(with: NSManagedObjectID). I hard-coded all the possible objectIds in an array and passed a random one each time.
Then, I used a simple fetch request with a NSPredicate that got passed a random UUID.
The difference in execution time is significant:
With ObjectID: 0.015282376s
With UUID: 1.093346287s
However, the strange thing is that the first method didn't actually produce any SQL queries (I logged them using the launch argument -com.apple.CoreData.SQLDebug 4). This would explain the speed but not why it doesn't need to communicate with the database at all.
I researched a bit but can't really figure out what object(with: NSManagedObjectID) actually does behind the scenes.
Does this mean, using a "custom" UUID property is not a good idea? I would really appreciate any insights on this!
I would not rely on the NSManagedObjectID in your code. It makes your code dependent on Apple's database implementation, which may change at any time, and it would not make your app resilient against future changes.
By way of example, you would not be able to use the new NSPersistentCloudKitContainer. It does not support NSManagedObjectID: see https://developer.apple.com/documentation/coredata/mirroring_a_core_data_store_with_cloudkit/creating_a_core_data_model_for_cloudkit
Instead of hardcoding NSManagedObjectID you are better off giving your entities unique UUIDs, as you have suggested. This may or may not affect performance, but you are better off in the long run, as the underlying core database technologies will shift.
You should just use a String to represent the NSManagedObjectID. To convert from NSManagedObjectID to string is easy:
objectID.uriRepresentation().absoluteString
To convert from String to NSManagedObjectID is slightly more complicated:
if let url = URL(string: viewModel.id),
let objectID = context.persistentStoreCoordinator?.managedObjectID(forURIRepresentation: url)
This will make your model objects cleaner.
NSManagedObjectID is good to be used within one application on one device, but it should never be stored and referenced across different applications on different device. I think it is not true that NSManagedObjectID is not supported for CloudKit.
As per why object(with: NSManagedObjectID) is fast. The document says it returns:
The identified object, if its known to the context; otherwise, a fault
with its objectID property set to objectID.
This means that if the object has been loaded before, it will return it immediately, if it has not been loaded before, it will return a fault. If you want to trigger a SQL to happen for a good comparison, you need to access one of the attributes after you call object(with: NSManagedObjectID). I would assume the performance should be very similar to the one using UUID.
I would like to hook an event after an object is retrieved from the database. This event should be fired regardless if the object is retrieved from .find() or via a .populate() call. I would think it should be similar to the other lifecycle callbacks like afterCreate or afterUpdate, etc.
Is this possible to do with Waterline and if so, how can it be accomplished?
To give an idea of what I'm trying to accomplish: I'm using Mongo to
store my data. There is the potential that the model schema has
changed since the record was last saved. I'd like to "upgrade" the
record retrieved from the database to the latest version using a
function defined on the model itself. I would like this process to
happen as transparently as possible after a query returns (ie. Not
have to explicitly call the function in each query callback function).
I am implementing data mapper in my zend framework 1.12 project and its working fine as expected. Now further more to enhance it i wants to optimize it in following way.
While fetching any data what id i wants to fetch any 3 field data out of 10 fields in my model table? - The current issue is if i fetches the only required values then other valus in domain object class remains blank and while saving that data i am saving while model object not a single field value.
Can any one suggest the efficient way of doing this so that i can fetch/update only required values and no need to fetch all field data to update the record.
If property is NULL ignore it when crafting the update? If NULLs are valid values, then I think you would need to track loaded/dirty states per property.
How do you go about white-listing the fields to retrieve when making the call to the mapper? If you can persist that information I think it would make sense to leverage that knowledge when going to craft the update.
I don't typically go down this path. I will lazy load certain fields on a model when it makes sense, but I don't allow loading parts of the object like this, rather I create an alternate object for use in rendering a list when loading the full object is too resource intensive. A generic dummy list object I just use with tabular data. It being populated from SQL or stored procedures result-sets, usually with my generic table mapper.
I'm using Play Framework and I have what I think is a very frequent persistence problem :
I display a form with values coming from the database and a field 'quantity'
The user updates the form and changes the 'quantity' value
He clicks on the "save button"
In the controller method called, I want to get the old value of 'quantity' and calculate the difference between the new and the old one before updating the DB
To make that, i use findById (before calling the object.save method), but it gives me the new value, not the old one : it apparently looks into some cache (which one ?) instead of requesting the DB
=> is that a normal ? how can i get the old value, make my calculation and then persist ?
Thanks a lot for your help, I do not want to manage old/new value in my DB...i'm sure it's bad practice !
UPDATE
public static void save(#Valid Lot lot) {
History element = new History();
element.date = new Date();
//HERE below it returns the new value, not the old one
Lot databaseLot = Lot.findById(lot.id);
element.delta= databaseLot.quantity - lot.quantity;
element.save();
lot.save();
list(null, null, null, null);
}
This is because, Play is doing some magic for you here.
When you pass a JPA Object into your controller, that contains an ID, Play will automatically retrieve this JPA Object from the database. If you look here, it explains this in a little more detail. It states (and assuming an action call that is passing in a User JPA Pojo)
You can automatically bind a JPA object using the HTTP to Java
binding.
You can provide the user.id field yourself in the HTTP parameters.
When Play finds the id field, it loads the matching instance from the
database before editing it. The other parameters provided by the HTTP
request are then applied. So you can save it directly.
So, how can you fix this? I guess the easiest way is to not pass the id as part of the Pojo Object, and to pass the ID as a separate parameter, therefore Play will believe the object is not required to be automagically retrieved from the database.
An alternative method, is to have a setter method for the quantity field, which updates the delta. So, Play will automatically retrieve the object from the DB, then call your setter method to update the values (as per normal POJO binding), and as part of that operation, your new quantity, and the delta are set. Perfect! This is the best option in my opinion as it also ensures that the business logic stays neatly inside of your Model, and not your Controller.
I can't speak to the Play Framework specifically, but in JPA, the EntityManager caches objects for it's lifetime, unless explicitly emptied. Because of that, querying for an object that the context is already managing will just give you the cached version. Furthermore, it sounds like the EM you are getting is declared as EXTENDED, which causes the same EM to be used across multiple requests (or perhaps the framework does the lookup under the covers and sets the values before you get to handle it).
You will either need to work around this cache or configure Play to use a TRANSACTION-scoped Persistence Context (aka EntityManager). I can't help you with the latter, but the former is easy enough.
int newQuantity = entity.getQuantity();
entityManager.refresh(entity);
// enity.getQuantity() should now give you the old value
// Do your calculation
// entity.setQuantity(newQuantity);
At the end of the transaction, your new state should be saved.
You may save quantity value in a hidden text and you can process that