Structuring/architecting an Azure Mobile Apps Service Entity Framework Code-First and .NET APK project - entity-framework

I am an experienced .NET/C# developer but new to pretty much all of the technologies/libraries here including SQL/DB work.
I am developing a project with an Azure/Entity Framework .NET backend and portable .Net APK for consumption in a number of other projects. I am trying to follow recommended practices and guidelines, but it's surprisingly hard to find documentation. I find myself repeatedly feeling like I'm fighting against the system, and slowly beating out a seemingly endless succession of fires with a blunt table spoon.
I find myself wondering if the overall architecture I'm using is the fundamental problem here. I prefer to pretend I'm not merely incompetent.
Current Structure
DTO contracts project
Interfaces for the DTO classes shared between the other two projects
Backend project
Implementations of the DTO interfaces + conversion to/from model classes
Code first database model classes
TableController<SOME_DTO_CLASS> implementations
ApiController for non-query operations
Portable SDK library project
Implementations of the DTO interfaces + conversion to/from SDK classes
SDK exposed classes for use from other applications
Service class that wraps MobileServiceClient and IMobileServiceTable and exposes SDK classes
Motivation/Implementation
Contract interfaces
The motivation for the DTO contract interfaces is to get as far away from magic strings / relying on member names as possible. These are interfaces rather than classes because TableController<T> requires implementations of ITableData, which is not available for use in the portable DTO contracts project.
Backend
The TableController<SOME_DTO_CLASS> classes GET methods currently refer to the current context (NOT this.Query()) and .Select() to create matching instances of the DTO classes. Lazy-loading is intact. These GET methods apply a .Where() with this.User to filter out only those entities the user has permission to access.
The Code-First model entirely derives from EntityData, even if the class is not going to be exposed via a TableController<T>. Navigation properties are used to types that are NOT exposed via their own TableController<T>. The fluent API is used to describe relationships.
The DTO classes expose their relation properties as the interface types rather than their concrete types because that's how interfaces work.
SDK
Currently this uses IMobileServiceTable but will likely switch over to IMobileServiceSyncTable at some point.
The DTO classes expose their relation properties as the interface types rather than their concrete types because that's how interfaces work.
Current flaming spoon target
Right now I've got the SDK successfully exposing it's own SDK types pulled down from the database. DB model -> DB DTO --> *MS Code* --> SDK DTO -> SDK exposed class all works.
Sort of.
The DB DTO classes' properties that expose other DB DTO classes appear to be ignored in transmission despite being part of the IQueryable returned in the GET method. I cannot retrieve them using $expand= as apparently The specified type member 'TestClass' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported. This still occurs if I switch from interface property types to concrete ones.
I could potentially avoid this issue by only including foreign key IDs and fetching linked entities separately in the SDK, but that seems highly inefficient and somewhat very nope.
Get to the question you 4AM fool!
Dis gud?
More specifically (and formally) is this current project structure reasonable and likely to be sustainable? Are there any obvious flaws or oversights that will prevent this from working?
Assuming this is reasonably reasonable, what is the proper way to tackle the DTO $expand issue?

The $expand attribute is the way to go, but unfortunately the Azure Mobile client SDK blocks this in the query string. It will be fixed in the future, but for right now your best bet is to use an attribute on the server side to add the query string on incoming requests.
For an example of this, see https://github.com/paulbatum/FieldEngineerLite/blob/master/FieldEngineerLite.Service/Helpers/ExpandPropertyAttribute.cs. The sample is for Azure Mobile Services, but that code can be easily applied to the Azure Mobile Apps server SDK.

Related

MVC Business Logic + DAL + Mapping ViewModel with EF Model

I'm starting to develop an application, and I want to know the best practices to organize the architecture of the solution.
Should I use EF Class Model as my ViewModel?
Should I put all my queries and db access in the model? or create a Service to manage all Db concerns ?
I'm using EF with DB First, because the db is already developed.
Thanks!
There are much more complete descriptions of application architecture out there, but here's the $.25 description.
EF Class Models are for your communication with a data store
Data Transfer Objects (DTO) are how modules communicate among themselves
(WebAPI to MVC, etc.)
ViewModels supply the data your UI requires
Look up "Separation of Concerns" as it pertains to application architecture, it can save your butt. Often, developers will dual-purpose these entities leading to some hilarious results when you find you've painted a corner for yourself. Not so funny if you are the "painter".
On the other hand, keeping these models requires extra effort and the mappings take CPU cycles. Here's a concrete example:
WebAPI accesses People entity (EF class) and maps to a PeopleDTO (not all fields, maybe additional information) and returns this to your MVC controller. MVC controller takes the PeopleDTO and merges it with supporting lookup tables (more WebAPI calls) to create a PeopleVM (ViewModel) that is used by your Razor Page.
In the scenario I just outlined, there are three different types of People object but each could have very different contents dependent on the needs of that "layer". Lots of tools exist to make the mapping less painful.
Clear?

Is it better to use POCO objects or detached EntityFramework object to expose database via WCF?

I created a WCF service in charge of exposing my database's data since I don't want the database to be directly accessed by my application (for security reasons) and i need to be able to share data with third-party applications.
My solution is structured this way: WPF application -> WCFService library -> DataAccessLayer library. (Arrows define assembly dependencies 'depends on')
To implement the WCF service I considered to simply return detached EntityFramework objects from the service BUT it forces the main application to have a dependency on the DataAccessLayer library.
The only way i can get around that is generating POCO objects and use them to send them over the wire, but now i have to map values back and forth EntityFramework.
At the moment i'm generating POCOs dynamically via a T4 template and I'm using AutoMapper to map values back and forth EntityFramework.
The Wcf service will just have to implement the repository pattern to expose data.
Is this a good solution? Are there any other option?
Is there any shortcoming i should be aware of?
Depending on your constraints, I would have to agree with this solution.
I created an almost identical solution, although our motivations were slightly different. Our client was Delphi Win32, and at the time didn't have good support for JSON, so we had to use SOAP.
The client also didn't support nullable primitives, so the POCOs removed all unsupported types, and performed other changes to ensure interoperability, then we used Automapper custom mappings to handle the two way conversions.
All the WCF services (contracts and implementations) where also generated by T4 templates, using a generic repository. With T4 templates, I was able to generate a separate WCF service per table for CRUD operations, and then manually created WCF services that were business specific.
Finally, I was also able to used T4 templates to generate the Delphi repositories that interacted with the SOAP services.
Or
You could just as easily move the POCOs (and code generation) to a separate project, change your DataAccessLayer library to reference the POCOs library and only contain the Db context made up of DbSets of your POCOs, and Data access logic but no entities (which are now POCOs). Your clients will not need to have a dependency on the DataAccessLayer library.
So... a good solution, depending on your constraints.

sharing DTO classes between server and client impossible because of EntityData : ITableData

I share a Data Transfer Object between an C# Azure Mobile Services server and client. I use the same class in both applications.
The TableController class used by Azure mobile services requires the DTO to inherit from 'EntityData', which in turn implements interface 'ITableData'.
ITable Data lives is part of reference:
Microsoft.WindowsAzure.Mobile.Service.Tables
I have not figured out how to include that reference without installing the entire server-side mobile services package in nuget:
WindowsAzureMobileServices.Backend
That includes OWIN, and many other references the client does not need. This is what I am doing currently. This works for a desktop application I am currently working on, but I think it will not work for universal apps and windows phone apps.
I also looked at microsoft's samples for mobile services, and there they use separate classes as DTOS for server and client.
Is it really the case that we have to write the same code twice?
No, but you could better make use of Shared Projects, and partial classes.
Your Shared Project will have common properties for the entities.
Other projects will reference this Shared one, and can add some other properties to shared entities, still using partial classes.
I have precise experience with AMS, so I know what you are meaning.
In my experience, is anyway not realistic to think to have exactly the same entity classes for client and server.
For instance, in so called Portable Class Libraries you can have a very small subset of framework, and references available.
Other than properties, you normally put attributes on POCO class files. On the client you may have some attributes that aren't available/meaningful for the server (e.g. SQLite attributes), or viceversa. You may can get stuck in this situation also with the shared projects approach I suggest, but it could be managed there with what so called preprocessor directives.

Is there a DataServiceContext.Set(type) equivalent to DbContext.Set(type)

I have recently created a pretty robust API built around Entity Framework's DbContext. I am using a lot of metadata programming and taking advantage of the fact that I can get my data with a call like DbContext.Set(typeof(Customer)). Only, in my API I do not know at compile time what type I will be passing to the Set method. This is working very well with EntityFramework and I would like to add another layer abstraction and have it work with both EntityFramework or DataServiceContext. So, I really have two questions.
Firstly, and more specifically, is there a DataServiceContext (i.e. odata/wcf) equivalent to the DbContext.Set(type) method?
Secondly, and more generally, is there a good resource that compares the APIs provided by DbContext with DataServiceContext?
EntityFramework and DataServices client API should not be mixed. Even though they look similar they are not. DbSet represents entity set. I don't think there is a strong contract around entity sets in DataServiceContext. Instead the name of the entity set is passed to methods that need to know this (e.g. look at DataServiceContext.AddObject() or DataServiceContext.CreateQuery() methods) as strings. In some sense it makes it much easier to program the DataServiceContext dynamically. On the other hand you still need to know what is on the other side of the pipe (i.e. the server). As said above WCF Data Services and EntityFramework are different technologies (even though they can work together) and their APIs, though similar, serve different purposes. Therefore comparing them would be like comparing apples to oranges.
The DbContext API in the client side is not the same from DbContext on server side. The main goal is to expose the data and model, which can be done pretty well. I think you may be overengeneering your app, since WCF Data Services can provide enough funcionalities.
Here is a link from Ladislav Mrnka, who is very good at entity framework, he shows how you could expose your robust api with WCF Data Services.
Implement WCF Data Service using the Repository Pattern

What specific issue does the repository pattern solve?

(Note: My question has very similar concerns as the person who asked this question three months ago, but it was never answered.)
I recently started working with MVC3 + Entity Framework and I keep reading that the best practice is to use the repository pattern to centralize access to the DAL. This is also accompanied with explanations that you want to keep the DAL separate from the domain and especially the view layer. But in the examples I've seen the repository is (or appears to be) simply returning DAL entities, i.e. in my case the repository would return EF entities.
So my question is, what good is the repository if it only returns DAL entities? Doesn't this add a layer of complexity that doesn't eliminate the problem of passing DAL entities around between layers? If the repository pattern creates a "single point of entry into the DAL", how is that different from the context object? If the repository provides a mechanism to retrieve and persist DAL objects, how is that different from the context object?
Also, I read in at least one place that the Unit of Work pattern centralizes repository access in order to manage the data context object(s), but I don't grok why this is important either.
I'm 98.8% sure I'm missing something here, but from my readings I didn't see it. Of course I may just not be reading the right sources... :\
I think the term "repository" is commonly thought of in the way the "repository pattern" is described by the book Patterns of Enterprise Application Architecture by Martin Fowler.
A Repository mediates between the domain and data mapping layers,
acting like an in-memory domain object collection. Client objects
construct query specifications declaratively and submit them to
Repository for satisfaction. Objects can be added to and removed from
the Repository, as they can from a simple collection of objects, and
the mapping code encapsulated by the Repository will carry out the
appropriate operations behind the scenes.
On the surface, Entity Framework accomplishes all of this, and can be used as a simple form of a repository. However, there can be more to a repository than simply a data layer abstraction.
According to the book Domain Driven Design by Eric Evans, a repository has these advantages:
They present clients with a simple model for obtaining persistence objects and managing their life cycle
They decouple application and domain design from persistence technology, multiple database strategies, or even multiple data sources
They communicate design decisions about object access
They allow easy substitution of a dummy implementation, for unit testing (typically using an in-memory collection).
The first point roughly equates to the paragraph above, and it's easy to see that Entity Framework itself easily accomplishes it.
Some would argue that EF accomplishes the second point as well. But commonly EF is used simply to turn each database table into an EF entity, and pass it through to UI. It may be abstracting the mechanism of data access, but it's hardly abstracting away the relational data structure behind the scenes.
In simpler applications that mostly data oriented, this might not seem to be an important point. But as the applications' domain rules / business logic become more complex, you may want to be more object oriented. It's not uncommon that the relational structure of the data contains idiosyncrasies that aren't important to the business domain, but are side-effects of the data storage. In such cases, it's not enough to abstract the persistence mechanism but also the nature of the data structure itself. EF alone generally won't help you do that, but a repository layer will.
As for the third advantage, EF will do nothing (from a DDD perspective) to help. Typically DDD uses the repository not just to abstract the mechanism of data persistence, but also to provide constraints around how certain data can be accessed:
We also need no query access for persistent objects that are more
convenient to find by traversal. For example, the address of a person
could be requested from the Person object. And most important, any
object internal to an AGGREGATE is prohibited from access except by
traversal from the root.
In other words, you would not have an 'AddressRepository' just because you have an Address table in your database. If your design chooses to manage how the Address objects are accessed in this way, the PersonRepository is where you would define and enforce the design choice.
Also, a DDD repository would typically be where certain business concepts relating to sets of domain data are encapsulated. An OrderRepository may have a method called OutstandingOrdersForAccount which returns a specific subset of Orders. Or a Customer repository may contain a PreferredCustomerByPostalCode method.
Entity Framework's DataContext classes don't lend themselves well to such functionality without the added repository abstraction layer. They do work well for what DDD calls Specifications, which can be simple boolean expressions sent in to a simple method that will evaluate the data against the expression and return a match.
As for the fourth advantage, while I'm sure there are certain strategies that might let one substitute for the datacontext, wrapping it in a repository makes it dead simple.
Regarding 'Unit of Work', here's what the DDD book has to say:
Leave transaction control to the client. Although the REPOSITORY will insert into and delete from the database, it will ordinarily not
commit anything. It is tempting to commit after saving, for example,
but the client presumably has the context to correctly initiate and
commit units of work. Transaction management will be simpler if the
REPOSITORY keeps its hands off.
Entity Framework's DbContext basically resembles a Repository (and a Unit of Work as well). You don't necessarily have to abstract it away in simple scenarios.
The main advantage of the repository is that your domain can be ignorant and independent of the persistence mechanism. In a layer based architecture, the dependencies point from the UI layer down through the domain (or usually called business logic layer) to the data access layer. This means the UI depends on the BLL, which itself depends on the DAL.
In a more modern architecture (as propagated by domain-driven design and other object-oriented approaches) the domain should have no outward-pointing dependencies. This means the UI, the persistence mechanism and everything else should depend on the domain, and not the other way around.
A repository will then be represented through its interface inside the domain but have its concrete implementation outside the domain, in the persistence module. This way the domain depends only on the abstract interface, not the concrete implementation.
That basically is object-orientation versus procedural programming on an architectural level.
See also the Ports and Adapters a.k.a. Hexagonal Architecture.
Another advantage of the repository is that you can create similar access mechanisms to various data sources. Not only to databases but to cloud-based stores, external APIs, third-party applications, etc.
You're right,in those simple cases the repository is just another name for a DAO and it brings only one value: the fact that you can switch EF to another data access technique. Today you're using MSSQL, tomorrow you'll want a cloud storage. OR using a micro orm instead of EF or switching from MSSQL to MySql.
In all those cases it's good that you use a repository, as the rest of the app won't care about what storage you're using now.
There's also the limited case where you get information from multiple sources (db + file system), a repo will act as the facade, but it's still a another name for a DAO.
A 'real' repository is valid only when you're dealing with domain/business objects, for data centric apps which won't change storage, the ORM alone is enough.
It would be useful in situations where you have multiple data sources, and want to access them using a consistent coding strategy.
For example, you may have multiple EF data models, and some data accessed using traditional ADO.NET with stored procs, and some data accessed using a 3rd party API, and some accessed from an Access database living on a Windows NT4 server sitting under a blanket of dust in your broom closet.
You may not want your business or front-end layers to care about where the data is coming from, so you build a generic repository pattern to access "data", rather than to access "Entity Framework data".
In this scenario, your actual repository implementations will be different from each other, but the code that calls them wouldn't know the difference.
Given your scenario, I would simply opt for a set of interfaces that represent what data structures (your Domain Models) need to be returned from your data layer. Your implementation can then be a mixture of EF, Raw ADO.Net or any other type of Data Store/Provider. The key strategy here is that the implementation is abstracted away from the immediate consumer - your Domain layer. This is useful when you want to unit test your domain objects and, in less common situations - change your data provider / database platform altogether.
You should, if you havent already, consider using an IOC container as they make loose coupling of your solution very easy by way of Dependency Injection. There are many available, personally i prefer Ninject.
The domain layer should encapsulate all of your business logic - the rules and requirements of the problem domain, and can be consumed directly by your MVC3 web application. In certain situations it makes sense to introduce a services layer that sits above the domain layer, but this is not always necessary, and can be overkill for straightforward web applications.
Another thing to consider is that even when you know that you will be working with a single data store it still might make sense to create a repository abstraction. The reason is that there might be a function that your application needs that your ORM du jour either does badly (performance), not at all, or you just don't know how to make the ORM bend to your needs.
If you are wrapping your ORM behind a well thought out repository interface, you can easily switch between different technologies as you see fit. It's not uncommon in my repositories to see some methods use EF for their work and others to use something like PetaPoco, or (gasp) ADO.net code. The repository abstraction enables you to use exactly the right tool for the job at hand without leaking these complexities into the client code.
I think there is a big misunderstanding of what many articles call "repository." And that's why there are doubts about what real value those abstractions bring.
In my opinion the repository in it's pure form is IEnumerable, while you and many articles are talking about "data access service."
I've blogged about it here.