We are writing a web service that needs to query a somewhat complex dataset from MS Dynamics. By "somewhat complex", I mean it needs to query multiple levels of the entity hierarchy. The Dynamics oData implementation limits the depth of oData queries to two (i.e. you can query a parent and a child - but not that child's child).
One way of moving forward would be to walk the entity hierarchy by making thousands of oData calls, but this is not at all appealing (kind of defeats the entire purpose of server side databases).
Another wrinkle is that we need to interact with many different Dynamics systems, none of which will allow us to load a custom .NET plugin. It probably doesn't matter, but we are writing our client-side code in Java, and really do need to use regular REST type calls.
I was thinking that if there were a way to predefine a data report in Dynamics that hits the complex query, that we could then maybe query that report via the REST interface. So far, I haven't found a way to do this, and I thought I'd toss this out to the community to see if anyone has any suggestions.
you can create a Query object in NAV and publish the Query Object as a Webservice
If the Query Object is not available in NAV (NAV version < 2013) you can create codeunit, generate the dataset in there and publish the codeunit as a webservice.
The third option is if the NAV is running on SQL server you can create a View/TempTable/Stored Procedure in there to create the dataset; or use SQL Analysis Services
Cheers!
We wound up dropping the REST API entirely and using SOAP instead. From there, we can do FetchXML and get what we need.
Related
I am new to domain driven design and trying to learn and implement in my project. My project structure up till now similar to this.
Maintainance Folder Maintainance.Data(Class
Library) Maintainance.Domain(Class Library)
Maintainance.Domin.Tests(test project)
MovieBooking Folder MovieBooking.Data(Class
Library) MovieBooking.Domain(Class Library)
MovieBooking.Domain.Tests(test project)
SharedKernel Common things
Web Application MovieBooking MVC Web
Application(which have reference to MovieBooking Domain)
In Maintainance boundned context I am keeping all CRUD, GetAll type things for say Movie, Country, Category, Subcategory entities in Maintainance DBContext.
Now in MovieBooking data layer I will also need to use these entities (mostly to display name or dropdown fills in view, kind of subset needed - not all properties needed, only few like Id, name)
There are few ways I can access this entities in Movie booking Bounded Context
Via web services - Need to create web api for common entities like Movie,Country,Category,Subcategory and call web api in web project (to fill Dropdowns or get name from entities)
Via Reference Context (Seperate Dbcontext) - Need to configure Dbset and then map a database view (with only require fields) to Dbset
Example :
modelBuilder.Entity().ToTable(ViewName);
For (1) it can be long term implmentation solution for me
(2) I have to create view (with only few properties) for each require table and it will increase my number of views in my DB drastically as I have enterprise level application.
Is there any other way I can achieve this? Anything I am missing in DDD to look for ?
Option 2, while it will save you time, is actually a very bad idea from the DDD perspective as it allows for violations of the transactional boundary guarantees that each aggregate is meant to enforce\represent.
Option 1 seems a better option, although there are still quite a bit of wiggle room for interpretation based on your brief description of your proposed solution. If I understood correctly, it is generally recommended to follow the below:
Do not expose your aggregate state directly since this exposes internals and increases coupling. Simple create meaningful DTO's and use something like Automapper to map your Aggregates to DTO's easilly and with little effort before sending it over.
Have a duplicate of the DTO definition in your client. This will reduce coupling and allow for easier deployments.
I strongly recommend reading the DDD orange book although I have to say that I cannot recall specifically on which chapter this is discussed. You will also benefit a lot by reading about hexagonal architecture (and I would search for that term in the orange book to find more info about your question).
There is actually one alternative that I can think of: if you're publishing events from your BC's you can create a workflow to translate the domain events to "public" events and then in the other BC listen for the public events that you need to and store the data that you need somewhere inside there. The difficulty of this ranges from very easy to quite problematic depending on your infrastructure. Be aware that it is not a very good idea to re-use your domain events for transmitting data to other BC's since this closely couples the two BC's.
I hope this helps. Please do not hesitate to elaborate if I did not understood the question well enough.
Reading a SO question, I realized that my Read services could provide some smarter object like ViewModels instead plain DTOs. This makes me reconsider what information should be provided by the objects returned by the Read Services
Before, using just DTOs, my Read Service just made flat view mapping of a database query into hash like structure with minimum normalization and no behavior.
However I tend to think of a ViewModel as something "smarter" that can have generated information not provided by the database, like status icon, calculated values, reformatted values, default values, etc.
I am starting to see that the construction of some ViewModel objects might get more complicated and has potential downsides if I made my generic ReadServiceInterface return ViewModels only:
(1) Should I plan some design restriction for the ViewModels returned by my CQRS? Like making sure that their construction is almost as fast as a plain DTO?
(2) DTOs by nature are easily serialized and ready to be sent to an external system in a SOA architecture or embedded into a message. Does this mean that using ViewModels will have a negative impact on my architecture?
(3) Which type of ViewModels should I keep outside my Read Services?
(4) Should I expect all ViewModels to be retrieved from Read Services?
In the past I implemented some ViewModels that needed more than one query. In a CQRS I suppose, that is a design smell, since everything they provide, should be in only one query.
I am starting a new project, where I thought that any query will return either aggregate objects or DTOs. Since now ViewModels come into play. I am wondering:
(5) Should I plan that queries within my architecture will yield two type of objects (ViewModels+Aggregates) or three (+DTO)?
View Models (VM) serve a single master: the View. We're usually consider the VM a pretty dumb object so in this regard, there's no technical difference between a VM and a DTO, only their purpose and semantics are different.
How you build a VM is an implementation detail. Some VM are pre generated and stored in a VM repository. Others are built in real-time by a service (or a query handler) either by querying the db directly or querying other repos/services then assembling the results. There's no right or wrong and no rules about how to do it. It comes down to preference.
In CQRS the important part is separation of commands from queries i.e more than one model. There's no rule about how many queries you should do or if you should return a view model or dto. As long as you have at least one read model dedicated for queries, it's CQRS.
Don't let technicalities complicate your design. Proper design is more about high level structure and not low level implementation. Use CQRS because having a read model simplifies your app, not for other reasons. Aim for simplification and clean code, not for rigid rules that dictate a 'how to' recipe.
I'm developing an Lighswitch application in VS2012 with an external SQL Server 2008. I'm using a WCF RIA Service to fetch my data, using these techniques: http://www.c-sharpcorner.com/UploadFile/raj1979/how-to-use-wcf-ria-service-in-lightswitch-2012/.
My main table contains some columns with static data and about 5 columns with foreign keys. The static data columns loads instant but all the foreign properties takes 3-5 secs to load on a screen with ~100 rows. This is not acceptable..
The problem for me is that I don't know where the bottleneck might be. Any ideas where I could start looking? The only thing I could think of so far is that my ADO.NET model uses Lazy Loading, and it does.
All the foreign key properties only has two columns with a PK and some text, so they're not data heavy.
Any help appreciated, thanks!
I don't know if this is the exact answer to your speed problem, but that article you quoted suggests that you add a new ADO.NET Data Model to the class library containing the RIA Service.
There's absolutely no reason to generate a second data model, as LightSwitch already generates one when you attach the external data source (your SQL Server database). You may as well re-use it. An EF data model takes time to construct, so I suppose that using two EF data models could explain the slowness you're experiencing, but it could also be something else entirely.
Have a look at this article (the original RIA Service/LightSwitch article, & still the best), which among several other techniques, Eric will show you how to re-use the data model that LightSwitch creates for your data source.
How Do I: Display a Chart Built On Aggregated Data
In this article, the RIA service is created to specifically aggregate data to display in a chart, but you can do whatever you need to do in it. It doesn't have to aggregate data. I use RIA services with an attached data source (a SQL database) extensively, to provide data for lists, reports, as well as lookup data for combo boxes.
I am almost a beginer at this new art, but as Yan says in his book, instead a very good one, lightswitch is directed to the screen. I dare to ask you if you need to have 100 lines in your screen. You can't even see them all. If you reduce that number to 20 you will reduce your waiting time to 1/5th. I've had the same problem and now I use the least possible number of rows in screen. For that to be possible without disturbing the business performance, I choose to open screens with the maximum number of parameters, not required, which I send when I open them. If one of the parameters is not the option in the case I mention it as "Nothing". And so, I can have a minimum number of lines in the grid, fullfiling the purpous of teh screen, the program performance and the business performance.
I am working on developing a set of assemblies that encapsulate parts of our domain that will be shared by many applications. Using the example of an order management system, one such assembly will contain all of the core operations an application can perform to/with an order. We are applying a simple version of CQS/CQRS so that all operations that change the state of the "system" are represented as public commands, such as CancelOrderCommand, ShipOrderCommand and CreateORderCommand. The command handlers are internal to the assembly.
The question I am struggling to answer is how to best expose the read model to consuming code?
The read model will be used by consuming code to perform queries. I don't know how all of the ways the read model will be used so the interface needs to be flexible to allow any query.
What complicates it for me is that I not only need to expose my aggregate root but there are also several "lookup" lists of related data that client applications may use. For example, each order has an associated OrderType which is data-driven (i.e., not an enum) and contains several properties that will drive some of our business rules that control what operations can/cannot be performed, etc. It is easy inside my module to manage this relationship; however, a client application that allows order creation will most likely need to display the list of possible OrderTypes to the user. As a result, I need to not only expose the list of Order aggregates but the supporting list of OrderTypes (and other lookup lists) from my read model.
How is this typically done?
I'm not sure what else to explain that will help trigger a solution, so please ask away...
I have never seen a CQRS based implementation expose a full dataset for ad-hoc querying so this is an interesting situation! In a typical CQRS scenario you would expose very specific queries because you may want to raise events when they are called (for caching for example - see this post for more details on that).
However since this is your design, let's not worry about "typical" or "correct" CQRS, I guess you just need a solution! One of the best new mechanisms for exposing data for flexible querying I have seen is the Open Data Protocol (OData). It will allow consumers to implement their own filtering, sorting and paging over a data source you expose.
Most implementations of this seem to deal with relational data. If you are dealing with a relational data source then OData might be a nice way to go. I suspect by your comment of "expose my aggregate root" that you might be using a document database? If so, there is one example I have seen of OData services on top of MongoDB: http://bloggingabout.net/blogs/vagif/archive/2012/10/11/mongodb-odata-provider-now-supports-arrays-and-nested-collections.aspx.
I hope that helps, OData is definitely worth looking into. It seems to be growing really quickly and is getting good support on both server and client technology platforms.
I'm working with a large hierarchical data set in sql server - modelled using the standard "EntityID, ParentID" kind of approach. There are about 25,000 nodes in the whole tree.
I often need to access subtrees of the tree, and then access related data that hangs off the nodes of the subtree. I built a data access layer a few years ago based on table-valued functions, using recursive queries to fetch an arbitrary subtree, given the root node of the subtree.
I'm thinking of using Entity Framework, but I can't see how to query hierarchical data like
this. AFAIK there is no recursive querying in Linq, and I can't expose a TVF in my entity data model.
Is the only solution to keep using stored procs? Has anyone else solved this?
Clarification: By 25,000 nodes in the tree I'm referring to the size of the hierarchical dataset, not to anything to do with objects or the Entity Framework.
It may the best to use a pattern called "Nested Set", which allows you to get an arbitrary subtree within one query. This is especially useful if the nodes aren't manipulated very often: Managing hierarchical data in MySQL.
In a perfect world the entity framework would provide possibilities to save and query data using this data pattern.
Everything IS possible with Entity Framework but you have to hack and slash your way in to it. The database I am currently working against has too many "holder tables" since Points for instance is shared with both teams and users. Both users and teams can also have a blog.
When you say 25 000 nodes do you mean navigational properties? If so I think it could be tricky to get the data access in place. It's not hard to navigate, search etc with entity framework but I tend to model on paper then create the database based on how I want to navigate while using entity framework. Sounds like you don't have that option.
Thanks for these suggestions.
I'm beginning to realise that the answer is to remodel the data in the database - either along the lines of nested sets as Georg suggests, or maybe a transitive closure table, which I've just come across.
That way, I'm hoping to get two key benefits:
a) faster querying aginst arbitrary subtrees
b) a data model which no longer requires recursive querying - so perhaps bringing it within easy reach of the Entity Framework!
It's always amazing how so often the right answer to a difficult problem is not to answer it, but to do something else instead!