I'm trying to get my head around NoSQL and Google Cloud Datastore, and I don't know how to chose between two different options for storing data.
I have a list of orders, and every order is for an unspecified number of products. What are the pros/cons of storing the product list as an array property for the order entity vs having product child entities for each order parent?
Firstly, be well aware of the distinction between the 2 possible approaches of implementing a relationship between entities:
one entity can contain a Key type property pointing to another entity (which might or might not exist!) - this is a functional relationship only, not one at the datastore level
having the 2 datastore entities in a parent-child (ancestry) relationship, inside the same datastore entity group.
Using the 2nd one has scalability implications, see also:
Ancestor relation in datastore
E-commerce Product Categories in Google App Engine (Python)
As for storing a list as an array property vs as separate entities, see Creating your own activity logging in GAE/P (where repeated properties is just how array properties are called in the ndb client library context).
Related
I am working on a POC (to move part of functionality from relational DB to cloud datastore). I have few questions:
I would need to refresh few "kind" every night as the data comes up
from a different data source (via flat files). I read about it, and
understood that there is not TRUNCATE kind of functionality in
datastore. I believe, only option is to retrieve the keys from the
"kind" in a loop and delete entity by entity. And use import functionality to load the new set of data. Is there any better
option?
Assume I have a kind called department, and a kind called
store. Now, I need a kind called dept-store. So for this parent
nodes are department and store. Is there a way to enforce this kind
of relationship? From the documentation I see that there can only be
one parent.
If i have a child entity in kind1 whose parent is
present in kind2, and they are linked together, is there a way to
query all the properties present in kind1 and kind2 together? From
relational DB perspective, it is like equi-join with "SELECT *". I
am looking for an equivalent functionality in datastore.
In order to answer your questions:
There is two ways to delete multiple entities. First, you can use Cloud Dataflow to delete entities in Bulk [1]. Second, once keys are retrieved you can make a batch delete operation by passing the keys to Datastore delete function, you have the usage example here [2]. In order to retrieve the keys you can run keys-only query [3].
In Datastore an entitiy can have only one parent but can have multiple children. But for your use case you may try to have a third kind, dept-store, and assign its properties as the keys of the entities from the department and the store kinds. This solution might need a good understanding of your neeeds for implementation, as Datastore by nature is Non-relational database.
You can lookup multiple entities providing the keys retrieved from kind1 and kind2 with batch operations [2].
Maybe, I'm a bit wrong, however, I'm trying to refactor my code right now via making use of #Converter annotation from JPA 2.1 to out-source the attribute-to-dbdata converting from the POJO class to a separate class. I'm mainly utilising a custom transformation for storing a kind of JSON blob into a database column. I have several cases, where I need to rely on the order of child entities, i.e., I store the set of utilised child entities in a many-to-many table to keep the relationship between the items and, furthermore, the order in a JSON array that just keeps the child entity identifiers (to keep the order). Then I have a resolving mechanism that keeps both sides always up-to-date, i.e., the db-data (string) will be converted to a (ordered) list of child entities (that are also stored in the DB and available via the set of child entities (many-to-many relationship).
So right now I'm wondering, whether I can handle this with a #Converter (AttributeConverter) implementation, since I'll require the set of child entities to resolve the db-data (string) to a (ordered) list of child entities (i.e. the "convertToEntityAttribute" method implementation)? Or whether I need to rely on my (a bit cumbersome) mechanism in the POJO class to convert between both sides?
AttributeConverter is for simple types only, not collections/maps, and as such provides a mapping between a java type and a database column. Some JPA implementations may allow mapping to multiple columns (I know the JPA implementation I use does, DataNucleus JPA, and some others may also allow it), but I doubt you'll get one that allows mapping to some other table entirely.
Better to look at your entity mappings and consider creating a dummy entity for this information somehow
We develop the back office application with quite large Db.
It's not reasonable to load everything from DB to memory so when model's proprties are requested we read from DB (via EF)
But many of our UIs are just simple lists of entities with some (!) properties presented to the user.
For example, we just want to show Id, Title and Name.
And later when user select the item and want to perform some actions the whole object is needed. Now we have list of items stored in memory.
Some properties contain large textst, images or other data.
EF works with entities and reading a bunch of large objects degrades performance notably.
As far as I understand, the problem can be solved by creating lightweight entities and using them in appropriate context.
First.
I'm afraid that each view will make us create new LightweightEntity and we eventually will end with bloated object context.
Second. As the Model wraps EF we need to provide methods for various entities.
Third. ViewModels communicate and pass entities to each other.
So I'm stuck with all these considerations and need good architectural design advice.
Any ideas?
For images an large textst you may consider table splitting, which is commonly used to split a table in a lightweight entity and a "heavy" entity.
But I think what you call lightweight "entities" are data transfer objects (DTO's). These are not supplied by the context (so it won't get bloated) but by projection from entities, which is done in a repository or service.
For projection you can use AutoMapper, especially its newer feature that I describe here. This allows you to reduce the number of methods you need to provide "for various entities" (DTO's), because the type to project to can be given in a generic type parameter.
This is a two part question, here is the situation:
I have an app that lists a set of product models. The user should be able to select from this list the product models that they specifically own. They can own more than 1 of the same type of product and should also be able to set a custom name for the products they own. So I set up my data model with two entities. Product, and OwnedProduct. OwnedProduct inherits from the Product entity and adds a customName property.
Question 1: Is this the best way to implement the model or should I be doing something with relationships? Right now I don't have any relationships in place.
The next question involves actually creating an OwnedProduct record from one of the Products that has been selected by the user.
Question 2: Is there a way to take the selected Product object, duplicate it , and then cast it as an OwnedProduct? Or would I have to create a brand new instance of OwnedProduct and then manually assign all of it's properties from the properties of the Product instance?
Question 1: Is this the best way to implement the model or should I be doing something with relationships? Right now I don't have any relationships in place.
Do you plan to have multiple owners within the same context?
Can a single product be owned by different owners with different custom names?
If yes I would take a look at relationships. I would not duplicate a product because of an ownership. If the product changes, it probably should change for all owners.
A possible datamodel for your requirements:
Question 2: Is there a way to take the selected Product object, duplicate it , and then cast it as an OwnedProduct? Or would I have to create a brand new instance of OwnedProduct and then manually assign all of it's properties from the properties of the Product instance?
As the clone will be a separate instance within your context you have to create a new one and copy all necessary properties. You can speed things up by iterating over NSEntityDescription properties and relationships. There is already a solution to cloning NSManagedObjects on SO
I have a database with a table for active orders and one for inactive orders. I would like to model this in Entity Framework as one entity called Orders. I also need a way to determine if an order in this collection is active or not, preferably by having a status property on the entity that is set according to what table it is in. Is there anyway to do this using Entity Framework 1. What about in Entity Framework 4?
You could create a view in your DB and generate the entity from that.
Take a look at the Table Per Concrete Type inheritance.
It is described here in ADO.NET Team Blog.
I think this is what you are looking for: How to: Define a Model with Multiple Entity Sets per Type (Entity Framework)
"The Entity Data Model (EDM) allows an entity type to be included by multiple entity sets within a single entity container or for an entity type to be included in entity sets in multiple entity containers. Defining multiple entity sets per type (MEST) allows users to streamline their code when databases have partitioning or other such scenarios where multiple tables have the same structure."
If I am understanding you correctly both active and inactive orders would share the same properties (for example: both would have a decimal "amount" property) if this is the case then in EF 1, I am pretty certain this is not possible. I think you will have to fall back to Mapping your entities to a POCO Orders object.
A good way to do one entity that shares multiple tables is to use Entity Splitting. MSDN has a very simple tutorial that walks you through the process which is very easy, however, you may need to reshape your data model: http://msdn.microsoft.com/en-us/data/jj715646.aspx