I want to use two different data sources in my Azure project:
a SQL Server that contains basic partial info regarding an item (allows indexable data and spatial search)
a Windows Azure Storage that contains full remaining info regarding an item (retrieved by key)
In this way I can combine the powerful of SQL Server with the easy scalability of Windows Azure Storage.
Imagine this Domain POCO class:
class Person
{
string Id { get; set; }
string Name { get; set; }
byte[] Picture { get; set; }
string Biography { get; set; }
}
I would like to use Entity Framework with fluent mapping to let EF understand that the properties Picture and Biography must be loaded from Windows Azure Storage (table, blob) instead of SQL Server (possibly Lazy loaded).
There's a way with EF (or NHibernate) to do this or I have to implement my own ORM strategy?
Thanks
I don't think you can let EF know about Azure storage but you can map only necessary properties to a specific table. For example,
modelBuilder.Entity<Person>().Ignore(p => p.Picture);
So assuming that you have a repository class for your Person class, what you want can be easily achieved by filling the repository class with Azure storage API and EF.
You're trying to solve this problem too early (at the DAL) in my opinion. Look at the web, it fetches large data (e.g. pictures) in a separate call to the server. That has scaled very well. The picture data is not included in the document itself for a reason, it would just slow everything down and it would not be very fault tolerant. If you put them together in one entity you've got the fast entity retrieval that is slowed down by your picture server as they both have to come together before leaving towards your business layer and finally towards the presentation layer. And in the business layer this data is probably just wasting memory (that's why you want to lazy load it). So I think you're making the decision too early. What you describe as your domain object looks like a domain object of the presentation layer to me, similar to a ViewModel. I'm not too big into domain driven design, but while there is a general model of your application, I assume that each part of your application will require a slightly different implementation of that model.
Regarding lazy loading, if you have that enabled and you attempt to send your object over the wire, even if Picture was not loaded, it will get serialized since the data contract serializer (or any other) will call get on your property.
That's probably not the answer you wanted, but I felt that I had to say this. Of course I am open to comments and criticism.
Related
Let's say you have a domain object:
class ArgumentEntity
{
public int Id { get; set; }
public List<AnotherEntity> AnotherEntities { get; set; }
}
And you have ASP.NET Web API controller to deal with it:
[HttpPost("{id}")]
public IActionResult DoSomethingWithArgumentEntity(int id)
{
ArgumentEntity entity = this.Repository.GetById(id);
this.DomainService.DoDomething(entity);
...
}
It receives entity identifier, load entity by id and execute some business logic on it with domain service.
The problem:
The problem here is with related data. ArgumentEntity has AnotherEntities collection that will be loaded by EF only if you explicitly ask to do so via Include/Load methods.
DomainService is a part of business layer and should know nothing about persistence, related data and other EF concepts.
DoDomething service method expects to receive ArgumentEntity instance with loaded AnotherEntities collection.
You would say - it's easy, just Include required data in Repository.GetById and load whole object with related collection.
Now lets come back from simplified example to reality of the large application:
ArgumentEntity is much more complex. It contains multiple related collections and that related entities have their related data too.
You have multiple methods of DomainService. Each method requires different combinations of related data to be loaded.
I could imagine possible solutions, but all of them are far from ideal:
Always load the whole entity -> but it is inefficient and often impossible.
Add several repository methods: GetByIdOnlyHeader, GetByIdWithAnotherEntities, GetByIdFullData to load specific data subsets in controller -> but controller become aware of which data to load and pass to each service method.
Add several repository methods: GetByIdOnlyHeader, GetByIdWithAnotherEntities, GetByIdFullData to load specific data subsets in each service method -> it is inefficient, sql query for each service method call. What if you call 10 service methods for one controller action?
Each domain method call repository method to load additional required data ( e.g: EnsureAnotherEntitiesLoaded) -> it is ugly because my business logic become aware of EF concept of related data.
The question:
How would you solve the problem of loading required related data for the entity before passing it to business layer?
In your example I can see method DoSomethingWithArgumentEntity which obviously belongs to Application Layer. This method has call to Repository which belongs to Data Access Layer. I think this situation does not conform to classic Layered Architecture - you should not call DAL directly from Application Layer.
So your code can be rewritten in another manner:
[HttpPost("{id}")]
public IActionResult DoSomethingWithArgumentEntity(int id)
{
this.DomainService.DoDomething(id);
...
}
In DomainService implementation you can read from repo whatever it needs for this specific operation. This avoids your troubles in Application Layer. In Business Layer you will have more freedom to implement reading: with serveral repository methods reads half-full entity, or with EnsureXXX methods, or something else. Knowledge about what you need to read for operation will be placed into operation's code and you don't need this knowledge in app-layer any more.
Every time situation like this emerged it is a strong signal about your entity is not preperly designed. As krzys said the entity has not cohesive parts. In other words if you often need parts of an entity separately you should split this entity.
Nice question :)
I would argue that "related data" in itself is not a strict EF concept. Related data is a valid concept with NHibernate, with Dapper, or even if you use files for storage.
I agree with the other points mostly, though. So here's what I usually do: I have one repository method, in your case GetById, which has two parameters: the id and a params Expression<Func<T,object>>[]. And then, inside the repository I do the includes. This way you don't have any dependency on EF in your business logic (the expressions can be parsed manually for another type of data storage framework if necessary), and each BLL method can decide for themselves what related data they actually need.
public async Task<ArgumentEntity> GetByIdAsync(int id, params Expression<Func<ArgumentEntity,object>>[] includes)
{
var baseQuery = ctx.ArgumentEntities; // ctx is a reference to your context
foreach (var inlcude in inlcudes)
{
baseQuery = baseQuery.Include(include);
}
return await baseQuery.SingleAsync(a=>a.Id==id);
}
Speaking in context of DDD, It seems that you had missed some modeling aspects in your project that led you to this issue. The Entity you wrote about looked not to be highly cohesive. If different related data is needed for different processes (service methods) it seems like you didn't find proper Aggregates yet. Consider splitting your Entity into several Aggregates with high cohesion. Then all processes correlated with particular Aggregate will need all or most of all data that this Aggregate contains.
So I don't know the answer for your question, but if you can afford to make few steps back and refactor your model, I believe you will not encounter such problems.
I have several concerns when trying to do DDD development with EF 4.2 (or EF 4.1) code first. I've done some extensive research but haven't come up with concrete answers for my specific concerns. Here are my concerns:
The domain cannot know about the persistence layer, or in other words the domain is completely separate from EF. However, to persist data to the database each entity must be attached to or added to the EF context. I know you are supposed to use factories to create instances of the aggregate roots so the factory could potentially register the created entity with the EF context. This appears to violate DDD rules since the factory is part of the domain and not part of the persistence layer. How should I go about creating and registering entities so that they correctly persist to the database when needed to?
Should an aggregate entity be the one to create it's child entities? What I mean is, if I have an Organization and that Organization has a collection of Employee entities, should Organization have a method such as CreateEmployee or AddEmployee? If not where does creating an Employee entity come in keeping in mind that the Organization aggregate root 'owns' every Employee entity.
When working with EF code first, the IDs (in the form of identity columns in the database) of each entity are automatically handled and should generally never be changed by user code. Since DDD states that the domain is separate from persistence ignorance it seems like exposing the IDs is an odd thing to do in the domain because this implies that the domain should handle assigning unique IDs to newly created entities. Should I be concerned about exposing the ID properties of entities?
I realize these are kind of open ended design questions, but I am trying to do my best to stick to DDD design patterns while using EF as my persistence layer.
Thanks in advance!
On 1: I'm not all that familiar with EF but using the code-first/convention based mapping approach, I'd assume it's not too hard to map POCOs with getters and setters (even keeping that "DbContext with DbSet properties" class in another project shouldn't be that hard). I would not consider the POCOs to be the Aggregate Root. Rather they represent "the state inside an aggregate you want to persist". An example below:
// This is what gets persisted
public class TrainStationState {
public Guid Id { get; set; }
public string FullName { get; set; }
public double Latitude { get; set; }
public double Longitude { get; set; }
// ... more state here
}
// This is what you work with
public class TrainStation : IExpose<TrainStationState> {
TrainStationState _state;
public TrainStation(TrainStationState state) {
_state = state;
//You can also copy into member variables
//the state that's required to make this
//object work (think memento pattern).
//Alternatively you could have a parameter-less
//constructor and an explicit method
//to restore/install state.
}
TrainStationState IExpose.GetState() {
return _state;
//Again, nothing stopping you from
//assembling this "state object"
//manually.
}
public void IncludeInRoute(TrainRoute route) {
route.AddStation(_state.Id, _state.Latitude, _state.Longitude);
}
}
Now, with regard to aggregate life-cycle, there are two main scenario's:
Creating a new aggregate: You could use a factory, factory method, builder, constructor, ... whatever fits your needs. When you need to persist the aggregate, query for its state and persist it (typically this code doesn't reside inside your domain and is pretty generic).
Retrieving an existing aggregate: You could use a repository, a dao, ... whatever fits your needs. It's important to understand that what you are retrieving from persistent storage is a state POCO, which you need to inject into a pristine aggregate (or use it to populate it's private members). This all happens behind the repository/DAO facade. Don't muddle your call-sites with this generic behavior.
On 2: Several things come to mind. Here's a list:
Aggregate Roots are consistency boundaries. What consistency requirements do you see between an Organization and an Employee?
Organization COULD act as a factory of Employee, without mutating the state of Organization.
"Ownership" is not what aggregates are about.
Aggregate Roots generally have methods that create entities within the aggregate. This makes sense because the roots are responsible for enforcing consistency within the aggregate.
On 3: Assign identifiers from the outside, get over it, move on. That does not imply exposing them, though (only in the state POCO).
The main problem with EF-DDD compatibility seems to be how to persist private properties. The solution proposed by Yves seems to be a workaround for the lack of EF power in some cases. For example, you can't really do DDD with Fluent API which requires the state properties to be public.
I've found only mapping with .edmx files allows you to leave Domain Entities pure. It doesn't enforce you to make things publc or add any EF-dependent attributes.
Entities should always be created by some aggregate root. See a great post of Udi Dahan: http://www.udidahan.com/2009/06/29/dont-create-aggregate-roots/
Always loading some aggregate and creating entities from there also solves a problem of attaching an entity to EF context. You don't need to attach anything manually in that case. It will get attached automatically because aggregate loaded from the repository is already attached and has a reference to a new entity. While repository interface belongs to the domain, repository implementation belongs to the infrastructure and is aware of EF, contexts, attaching etc.
I tend to treat autogenerated IDs as an implementation detail of the persistent store, that has to be considered by the domain entity but shouldn't be exposed. So I have a private ID property that is mapped to autogenerated column and some another, public ID which is meaningful for the Domain, like Identity Card ID or Passport Number for a Person class. If there is no such meaningful data then I use Guid type which has a great feature of creating (almost) unique identifiers without a need for database calls.
So in this pattern I use those Guid/MeaningfulID to load aggregates from a repository while autogenerated IDs are used internally by database to make a bit faster joins (Guid is not good for that).
I'm trying to understand the best architecture for my MVC2 site.
As I have been experimenting with getting the data in and out of a database with Entity Framework, I am beginning to realize the simple domain-models I have so far constructed do not map to all the needs of my planned views. So I am considering following the accpepted answer to this question: Why Two Classes, View Model and Domain Model?.
But there seems to be redundancy with little payoff that I can perceive between the domain-models and the EF models, and I can't even hardly understand the conceptual difference. I do NOT have as a requirement the need to switch data sources down the road, and I do not forsee the need to switch my ORM solution either.
QUESTION:
If I follow this pattern then, since I am using Entity Framework, shouldn't I just use my EF entities to serve directly as the domain models? (note: I haven't thought through the "how" of that, but answers there are welcome too.) Or am I still advised to manage a separate set of domain-models?
It seems you've got some redundancy here. Reading your paragraph:
But there seems to be redundancy with
little payoff that I can perceive
between the domain-models and the EF
models, and I can't even hardly
understand the conceptual difference.
I would argue that there is no real difference between the EF Model and your Domain Model. In the projects I create, my EF Model is my Domain model.
However, my Domain model classes are not the same as my ViewModels. The Domain model class might contain data that is not interesting for the View, or maybe the view needs information that is calculated/evaluated based on information in view. A simple example might be:
public class Session // Domain model (and EF Model
{
public int Id {get; set; }
public DateTime Start {get; set; }
public int DurationInMinutes {get; set; }
}
public class SessionViewModel // The viewmodel :p
{
public DateTime Start {get; set; }
public int DurationInMinutes {get; set;}
public DateTime End
{
get
{
return Start.Add(TimeSpan.FromMinutes(DurationInMinutes));
}
}
}
In this example I'm interested in displaying the actual End-time in my View, but I have no interest in storing it in the database, as that might lead to data-discrepencies (DurationInMinutes + Start might not equal End if data is corrupted upon saving)
When I first started coding this way, I ended up doing alot of manual work mapping my Domain models to ViewModels, and back. AutoMapper changed all that :) Google it, or NuGet it and it will make your life a whole lot easier :)
Hope this helps a little. Please comment if I'm totally missing the point :)
Update to address the comment
DataAnnotations would then be applied to the ViewModel, because normally DataAnnotations denote how the data should be displayed and validated in the View.
For instance you would put the [Required] attribute on public DateTime Start {get; set;} in order for the Html.DisplayFor extensions automatically validates your HTML according to your dataannotations.
By definition (by some anyway) the Domain Model should not contain any code or logic related to your business logic. The Domain Model is simply responsible for containing the data pretty raw according to your datastore. Personally I like to put some sort of Service layer inbetween that is responsible for fetching the data and returning ViewModels, and also doing the reverse.
The ultimate goal is to avoid referencing your domainmodel directly from your controllers.
Of course, all these points has to be weighed in reference to the size of the project. It's certainly overkill to do all this just to mock up a test-site - but in any other project where you'll actually be deploying something that might scale, expand or otherwise change, it's a good practice to get used to, as it seriously increases your ability to do so.
Another key point to this approach is that you are forced to abstract your operations down to smaller and more managable units, enabling better and more precise unit-tests.
I am working on a project using entity framework. Is it okay to use partial classes of the EF generated classes as the business layer. I am begining to think that this is how EF is intended to be used.
I have attempted to use a DTO pattern and soon realized that i am just creating a bunch of mapping classes that is duplicating my effort and also a cause for more maintenance work and an additional layer.
I want to use self-tracking-entities and pass the EF entities to all the layers. Please share your thoughts and ideas. Thanks
I had a look at using partial classes and found that exposing the database model up towards the UI layer would be restrictive.
For a few reasons:
The entity model created includes a deep relational object model which, depending on your schema, would get exposed to the UI layer (say the presenter of MVP or the ViewModel in MVVM).
The Business logic layer typically exposes operations that you can code against. If you see a save method on the BLL and look at the parameters needed to do the save and see a model that require the construction of other entities (cause of the relational nature the entity model) just to do the save, it is not keeping the operation simple.
If you have a bunch of web services then the extra data will need to be sent across for no apparent gain.
You can create more immutable DTO's for your operations parameters rather than encountering side effects cause the same instance was modified in some other part of the application.
If you do TDD and follow YAGNI then you will tend to have a structure specifically designed for the operation you are writing, which would be easier to construct tests against (not requiring to create other objects not realated to the test just because they are on the model). In this case you might have...
public class Order
{ ...
public Guid CustomerID { get; set; }
... }
Instead of using the Entity model generated by the EF which have references exposed...
public class Order
{ ...
public Customer Customer { get; set; }
... }
This way the id of the customer is only needed for an operation that takes an order. Why would you need to construct a Customer (and potentially other objects as well) for an operation that is concerned with taking orders?
If you are worried about the duplication and mapping, then have a look at Automapper
I would not do that, for the following reasons:
You loose the clear distinction between the data layer and the business layer
It makes the business layer more difficult to test
However, if you have some data model specific code, place that is a partial class to avoid it being lost when you regenerate the model.
I think partial class will be a good idea. If the model is regenerated then you will not loose the business logic in the partial classes.
As an alternative you can also look into EF4 Code only so that you don't need to generate your model from the database.
I would use partial classes. There is no such thing as data layer in DDD-ish code. There is a data tier and it resides on SQL Server. The application code should only contain business layer and some mappings which allow persisting business objects in the mentioned data tier.
Entity Framework is you data access code so you shouldn't built your own. In most cases the database schema would be modified because the model have changed, not the opposite.
That being said, I would discourage you to share your entities in all the layers. I value separation of UI and domain layer. I would use DTO to transfer data in and out of the domain. If I have the necessary freedom, I would even use CQRS pattern to get rid of mapping entities to DTO -- I would simply create a second EF data access project meant only for reading data for the UI. It would be built on top of the same database. You read data through read (anemic -- without business logic) model, but you modify it by issuing commands that are executed against real model implemented using EF and partial methods.
Does this answer your question?
I wouldn't do that. Try too keep the layers independent as possible. So a tiny change in your database schema will not affect all your layers.
Entities can be used for data layer but they should not.
If at all, provide interfaces to be used and let your entities implement them (on the partial file) the BL should not know the entities but the interfaces.
I'm building the 2nd iteration of a web-based CRM+CMS for a franchise service business in ASP.NET MVC 2. I need to control access to each franchise's services based on the roles a user is assigned for that franchise.
4 examples:
Receptionist should be able to book service jobs in for her "Atlantic Seaboard" franchise, but not do any reporting.
Technician should be able to alter service jobs, but not modify invoices.
Managers should be able to apply discount to invoices for jobs within their stores.
Owner should be able to pull reports for any franchises he owns.
Where should franchise-level access control fit in between the Data - Services - Web layer?
If it belongs in my Controllers, how should I best implement it?
Partial Schema
Roles class
int ID { get; set; } // primary key for Role
string Name { get; set; }
Partial Franchises class
short ID { get; set; } // primary key for Franchise
string Slug { get; set; } // unique key for URL access, eg /{franchise}/{job}
string Name { get; set; }
UserRoles mapping
short FranchiseID; // related to franchises table
Guid UserID; // related to Users table
int RoleID; // related to Roles table
DateTime ValidFrom;
DateTime ValidUntil;
Controller Implementation
Access Control with [Authorize] attribute
If there was just one franchise involved, I could simply limit access to a controller action like so:
[Authorize(Roles="Receptionist, Technician, Manager, Owner")]
public ActionResult CreateJob(Job job)
{
...
}
And since franchises don't just pop up over night, perhaps this is a strong case to use the new Areas feature in ASP.NET MVC 2? Or would this lead to duplicate Views?
Controllers, URL Routing & Areas
Assuming Areas aren't used, what would be the best way to determine which franchise's data is being accessed? I thought of this:
{franchise}/{controller}/{action}/{id}
or is it better to determine a job's franchise in a Details(...) action and limit a user's action with [Authorize]:
{job}/{id}/{action}/{subaction}
{invoice}/{id}/{action}/{subaction}
which makes more sense if any user could potentially have access to more than one franchise without cluttering the URL with a {franchise} parameter.
Any input is appreciated.
Edit:
Background
I built the previous CRM in classic ASP and it runs the business well, but it's time for an upgrade to speed up workflow and leave less room for error. For the sake of proper testing and better separation between data and presentation, I decided to implement the repository pattern as seen in Rob Conery's MVC Storefront series.
How to arrange services and repositories?
It makes sense to have a JobService that retrieves any service jobs based on available filters, eg. IQueryable<Job> GetJobs();. But since a job can only belong to one franchise, a function like IQueryable<Job> GetJobs(int franchiseID); could belong in either FranchiseService or in JobService. Should FranchiseService act as a CatalogService (like in MVC Storefront)?
Let me take a stab at answering this. I am in the process of playing with a sample app that touches some of the aspects mentioned. This is not an authoritative answer, merely experience.
Where should franchise-level access control fit in between the Data - Services - Web layer?
This access restrictions should
permeated through your application at
two levels 1) the database 2) the
application layer. In an MVC context I
would suggest having creating a custom
Authorization attribute - this handles
the security between the Web-Services
layer. I would have this attribute do
two things
Get the current roles allowed for the user (either from the DB of it may
be stored in the user session)
Do the checking to see if the user is part of the allowed list of roles.
With regards to the database, this
depends on how you are storing the
data, one database for all franchises
or database per franchise. In the
first case there are several ways to limit
and setup access restrictions for
data to a particular
franchise.
Since franchises don't just pop up over night, perhaps this is a strong case to use the new Areas feature in ASP.NET MVC 2? Or would this lead to duplicate Views?
I think that Areas should be used to
split and group functionality. If you
were to use Areas to split franchises,
this is where I see a duplication of
views, controllers etc. occurring. Duplicate
views can be overcome by using a
custom view engine to specifically
overriding the way MVC locates your
views. Plug: See my answer to ASP.NET MVC: customized design per domain
Assuming Areas aren't used, what would be the best way to determine which franchise's data is being accessed?
As mentioned above, you could the
users session to store basic
information such as the franchise the
user belongs to and the roles etc
assigned. I think the rule I read
somewhere goes along the lines of
"Secure your actions, not your
controllers"
Create you routes etc for the norm and
not for the exception. eg. Is there
currently a business case that says a
user can have access to more than one
franchise?
How to arrange services and repositories?
Have a set of base services or base
classes that will contain all the
information required for a particular
franchise such as the franchiseId.
Th main issue that it does resolve is
that your service methods are cleaner
not having the franchiseId argument.
The repository however may need this
value since as some point you need to
disambiguate the data you are
requesting or storing (assuming one db
for all franchises). However, you
could overcome some of this using IoC.
The downside I see is that
they there will always be calls to the
database every time your objects are
creating (i.e. if the franchise
route were to be used, you would need
to go the database to obtain the
corresponding franchiseId every time
you create a service object. ( I might
be mistaken on this one, since the IoC
containers do have some LifeStyle
options that may be able to assist and
prevent this) You could have
a list of Franchises that are created
on you Application start that you
could use to map your route values to
obtain the correct information. This
part of the answer is scattered, but
the main thing is that IoC will help
you decouple a lot of dependencies.
Hope this helps..