We would like to create an OData REST API. Our data model is such that each customer has their own database. All database objects have the same definition across all customer databases, with the exception of a single table.
The customer specific table we will call Contact. When a customer adds a column the system creates a column with a standardised name with a definition translated from options selected by the user in the UI. The user only refers to the column data by a field name they have specified to enable the user to be able to generate friendly queries.
It seems to me that the following approaches could be used to enable OData for the model described:
1) Create an OData open type to cater for the dynamic properties. This has the disadvantage of user requests for a customer not providing an indication of the dynamic properties that can be queried against. Even though they will be known for the user (via token authentication). Also, because dynamic properties are a dictionary, some data pivoting and inefficient query writing would be required. Not sure how to implement the IQueryable handling of query options for the dynamic properties to enable our own custom field querying.
2) Create a POCO class with e.g. 50 properties; CustomField1, CustomField2... Then somehow control which fields are exposed for use in OData calls. We would then include a separate API call to expose the custom field mapping. E.g. custom field friendly name of MobileNumber = CustomField12.
3) At runtime, check to see if column definitions of table changed since last check. If have, generate class specific to customer using CodeDom and register it with OData. Aiming for a unique URL for each customer. E.g. http://domain.name/{customer guid}/odata
I think the ideal for us is option 2. However, the fact the CustomField1 could be an underlying SQL data type of nvarchar, int, decimal, datetime, etc, there are added complications.
Has anyone a working example of how to achieve what has been described, satisfactorily?
Thanks in advance for any help.
Rik
We have run into a similar situation but with our entire dataset being unknown until runtime. Using the ODataConventionModelBuilder and EdmModel classes, you can add properties dynamically to the model at runtime.
I'm not sure whether you will have to manually add all of the properties for this object type even though only some of them are unknown or whether you can add your main object and then add your dynamic ones afterwards, but I guess either would be workable.
If you can get hold of which type of user it is on the server, you could then add only the properties that you are interested in (like option 3 but not having to CodeDom).
There is an example of this kind of untyped OData server in the OData samples here that should get you started: https://github.com/OData/ODataSamples/tree/master/WebApi/v4/ODataUntypedSample
The research we carried out actually posed Option 1 as the most suitable approach for some operations. i.e. Create an SQL view that unpivots the data in a table to a key/value pair of column name/column value for each column in the table. This was suitable for queries returning small datasets. This was far less effort than Option 3 and less confusing for the user than Option 2. The unpivot query converted the field values to nvarchar (string) values and thus meant that filtering in the UI by column value data types was not simple to achieve. (If we decide to implement this ability, I believe this can be achieved by creating a custom attribute that derives from EnablQueryAttribute, marking the controller action with it and manipulate the IQueryable before execution).
However, we wanted to expose a /Contacts/Export endpoint that when called would output the columns from a table with a fixed schema joined on a table with a client specific schema and output to a CSV file. All the while utilising the OData supported filter syntax. One of our customer databases has more than 12 million rows of data and is made up of approximately 30 columns.
To achieve this it looks like our best bet would have been to work with the Microsoft.OData.Core.UriParser.UriQueryExpressionParser class, unfortunately Microsoft in their wisdom have declared this as internal, as well as many of it's dependants.
Walking an abstract syntax tree built from OData supported query options and applying our own visitor to each node to build some dynamic Linq query/SQL seems like a possible solution.
For the time-being we will simply implement a cut-down set of supported $filter criteria without the support for grouping parenthesis.
Related
I am working on new API, but I am stuck with the choice for enabling Projection support.
my Entity User contains some properties and have some relationships too.
What should be the preferred approach for API?
Approach1
myApi.com/User/Fields="Id,Name,Email"
it returns my UserDTO with just populating Id,Name,Email and rest properties are null (I can further ignore the null before serializing JSON to send over the wire)
Approach2
myApi.com/User/GetCustomPropertiesABC"
it returns my UserCustomABCDTO . This Model contains just three properties for Id, Name, Email.
My question
I personally feel Approach 1 gives more Reuse quotient to API, Every time new selector required in Approach 2 will need to create a New DTO, But what is standard as per REST principles?
well, you could use ODATA for this, it would be perfect for it actually : http://www.odata.org/
Or, you could create an endpoint which accepts a request like your Approach 1 and returns dynamic. This way you don't have to worry about creating DTOs, you just create a new dynamic object, add whatever properties are required and return just that.
This keeps the returned entity small as it avoids who knows how many null or empty properties which don't really add anything useful.
In a JHipster based project, we need to selectively filter out certain columns based on role/user logged in. All users will be able to view/modify most of the columns, but only some privileged users will be able to view/modify certain secure fields/columns.
It looks like the only option to get this done is using EntityListeners. I can use an EntityListener and mask a certain column during PostLoad event. Say for example, I mask the column my_secure_column with XXX and display to the user.
User then changes some other fields/columns (that he has access to) and submits the form. Do I have to again trap the partially filled in entity in PreUpdate event, get the original value for my_secure_column from database and set it before persisting?
All this seems inefficient. Scoured several hours but couldn't find a specific implementation that best suits this use case.
Edit 1: This looks like a first step to achieving this in a slightly better way. Updating Entities with Update Query in Spring Data JPA
I could use specific partial updates like updateAsUserRole, updateAsManagerRole, etc., instead of persisting the whole entity all the time.
#Repository
public interface CompanyRepository extends JpaRepository<Company, Integer> {
#Modifying(clearAutomatically = true)
#Query("UPDATE Company c SET c.address = :address WHERE c.id = :companyId")
int updateAddress(#Param("companyId") int companyId, #Param("address") String address);
}
Column based security is not an easy problem to solve, and especially in combination with JPA.
Ideally you like to avoid even loading the columns, but since you are selecting entities this is not possible by default, so you have to remove the restricted content by overriding the value after load.
As an alternative you can create a view bean (POJO) and then use JPQL Constructor Expression. Personally I would use CriteriaBuilder. construct() instead of concatenating a JPQL query, but same principle.
With regards to updating the data, the UI should of cause not allow the editing of restricted fields. However you still have to validate on the backend, and I would recommend that you check if the column was modify before calling JPA. Typically you have the modifications in a DTO and would need to load the Entity anyway, if a restricted column was modified, you would send an error back. This way you only call JPA after the security has been checked.
I am developing a SPA using Angular-Breeze-WebAPI-EntityFramework.
Now Breeze uses the Entity Framework metadata information to create it's own Breeze models. We use this in our application for Breeze validation.
So far, it's all been nice and easy. Now we are having to create a search page (say for querying customers). The search can be by Customer.Name or by Product.Id (which would return a list of customers who have bought that product). The result is a ng-repeater, which displays Customer.Name, Order.LastPlaced etc.
if you are getting confused by the tables and columns, forget that. What I am only trying to get to is that, both the search object and the result object are not 1:1 with Entity tables (or objects). So obviously I feel the need to create a custom object (one for the search and one for the results). My question primarily is where and how do I create that object?
If I create it at the data layer, Breeze would have no idea of the metadata for each of the properties (since it uses EF for that).
I obviously can't create just a JavaScript object, since I will have to query the database (using EF) to search and populate the object.
So where does one create such a custom object (traversing multiple tables) such that Breeze still can figure out the metadata and perform validation and such when the need arises?
Thank you all.
You can create metadata on the client for types that the server either doesn't know about or doesn't have the schema for. See http://www.breezejs.com/documentation/metadata-by-hand.
I am implementing data mapper in my zend framework 1.12 project and its working fine as expected. Now further more to enhance it i wants to optimize it in following way.
While fetching any data what id i wants to fetch any 3 field data out of 10 fields in my model table? - The current issue is if i fetches the only required values then other valus in domain object class remains blank and while saving that data i am saving while model object not a single field value.
Can any one suggest the efficient way of doing this so that i can fetch/update only required values and no need to fetch all field data to update the record.
If property is NULL ignore it when crafting the update? If NULLs are valid values, then I think you would need to track loaded/dirty states per property.
How do you go about white-listing the fields to retrieve when making the call to the mapper? If you can persist that information I think it would make sense to leverage that knowledge when going to craft the update.
I don't typically go down this path. I will lazy load certain fields on a model when it makes sense, but I don't allow loading parts of the object like this, rather I create an alternate object for use in rendering a list when loading the full object is too resource intensive. A generic dummy list object I just use with tabular data. It being populated from SQL or stored procedures result-sets, usually with my generic table mapper.
I am trying to create WCF Data service project using Entity Framework. ( I am new to both).
I created entities using DB.
Now, I created service operation, which returns the IQueryable<entity>.
My problem is
I do not want to return the entire set of columns in the entity. I cannot delete them from the entity as it is not null. How to avoid these?
I have few FK columns and I need other column details of the table. How to include columns from other table?
Why it is not possible to use POCO class to be returned from WCF Data service?
How do I format the response; i.e., add few more details to the response like page number etc, change the xml tags, remove few details like "link rel"?
I have tried a lot of things to achieve 1 and 2. But finally I realised that I can only use the entity as it is to make it work.
I have no idea about 4.
Any suggestions would be appreciated.
1. I do not want to return the entire set of columns in the entity.
2. I have few FK columns and I need other column details of the
table.
For this, you should define a new class that matches what you need / what you want your clients to see. That can be a straight POCO class - no special requirements. Assemble that class for each entity, leaving out the unwanted columns, and grabbing the extra field or two for the FK columns into that new class. Return an IQueryable<YourNewClass> instead of the entity class directly.
To avoid huge left-right-assignment statements just to fill the properties of the new class, you should have a look at AutoMapper which makes it really easy to copy around classes that are very similar to one another (e.g. missing or adding a few properties).
4. How do I format the response; i.e., add few more details to the
response like page number etc, change the xml tags, remove few details
like "link rel"?
That's not possible - the OData protocol very strictly defines what's going to be in the message, what links there are etc. If you can't live with that - you'll have to roll your own WCF REST service and drop the WCF Data Service stuff altogether.
Check out the WCF REST Developer Center on MSDN if you want to investigate that route more thoroughly.
Update: that link seems to have gone dead - try WCF Web Http Programming Overview instead.
Make sure you have an Id property or you specify either [Key] or [DataServiceKey("Your_Custom_ID_Property")]
For me this sorted out the issue