Wondering how some of the more experienced (or anyone with a better idea than I have) would tackle my particular modeling scenario...
I have a typical "Category -> SubCategory ->TertiarySubCategory" scenario and I'm not sure if I'm mapping it out correctly. I am mapping this directly to an MVC route since raven seems to lend itself well with this. Under the final category (which can be at the first, 2nd or 3rd levels there will be a list of items associated with only that level of a category. So, we might have something like:
Single level category: '/Politics/'
Second level category: 'Politics/People' or 'Politics/Websites'
Tri-Level Category: 'Sports/Pro/Volleyball' or 'Sports/College/Football'
In a traditional RDBMS this is easy through primary/foreign keys + a few joins... so, wondering how I would handle with Raven?
From what I have read should I store the entire 'sports/pro/volleyball' URI or Key in a list of items that fall under it?
i.e. -
public class CategoryItem
{
public string FriendlyName {get;set;} // Volleyball or Pro Volleyball
public string CategoryURI {get;set;} // i.e. - "/sports/pro/volleyball/"
public string content {get;set;} // i.e. - "Who is the best Pro Volleyball Athlete?"
public List<string> Comments {get;set;}
}
// then we could store something like this:
var survey1 = new CategoryItem();
survey1.CategoryURI = "/sports/pro/volleyball/"
survey1.Content = "Who is the best female pro volleyball player?";
survey1.Comments.Add(new Comment("Misty May"));
var survey2 = new CategoryItem();
survey2.CategoryURI = "/sports/pro/volleyball/";
survey2.Content = "Who is the best male pro volleyball player?";
survey2.Comments.Add(new Comment("Some guy I don't kow");
// asuumes ravenSession was alreadyopened...
ravenSession.Store(survey1);
ravenSession.Store(survey2);
ravenSessoin.SaveChanges();
//{ ...... etc ..... }
//Then I can query by CategoryURI without needing joins (denormalization).... i.e. -
var items = session.Query<CategoryItem>()
.Where(x => x.CategoryURI == "/sports/pro/volleyball/");
Or should I create a List items member of the actual category class? Each item would have a list of it's own comments... meaning everything's stored in a single document within Raven - i.e. -
public class Category
{
public string FriendlyName {get;set;} // i.e. - "Volleyball" or "Pro Volleyball"
public string URI {get;set;} // i.e. - "/sports/pro/volleyball" which is the MVC path
public List<CategoryItem> Items {get;set;}
}
public class CategoryItem
{
public string Content {get;set;}
public List<string> Comments {get;set;}
}
var vballCat = new Category();
vballCat.FriendlyName = "Pro Volleyball";
vballCat.URI = "/sports/pro/volleyball/"; // equivalent to the MVC route
var catItem = new CategoryItem().
catItem.Content = "Who is the best male pro volleyball player?";
catItem.Comments.Add("Misty May");
catItem.Comments.Add("Some Guy 1");
vballCat.Items.Add(catItem);
ravenSession.Store(vballCat);
ravenSession.SaveChanges();
..... now once I pull the primary cat i.e. - "/sports/pro/volleyball/" I have everything I need already under it
var items = session.Query<Category>()
.Where(x => x.URI == "/sports/pro/volleyball/");
{ ............. etc ............... }
Now here I can just iterate through the Items collection and it's collection of comments.... does this use eager loading? What if I had a million comments under one category item? When I load the main category would it load all one million comments too!?!?
I would appreciate any help you can provide. Sorry if this example/question is unclear... I'll try to clarify anything if you guys need it. Thanks again!
The answer is that it depends on the size of your data and your usage scenario.
The first example is useful if you have large number of items and want to access categories without its items.
The second example is useful if you usually access category with its items, and the size of items is limited (note that limited is still high, several thousands wouldn't cause me to blink).
Note that there is no such thing as eager / lazy loading in RavenDB, you are talking about a single document vs. multiple documents, not about relations between documents. The entire document is loaded when you need it.
Another thing to remember is that it is usually faster to query by id than querying. That means that if you have ids that already looks very much like Document Ids, you might as well MAKE them the document ids.
Related
I have the following in Entity Framework Core:
public class Book {
public Int32 Id { get; set; }
public String Title { get; set; }
public virtual Theme Theme { get; set; }
}
public class Theme {
public Int32 Id { get; set; }
public String Name { get; set; }
public Byte[] Illustration { get; set; }
public virtual ICollection<Ebook> Ebooks { get; set; }
}
And I have the following linq query:
List<BookModel> books = await context.Books.Select(x =>
new BookModel {
Id = x.Id,
Name = x.Name,
Theme = new ThemeModel {
Id = x.Theme.Id,
Name = x.Theme.Name
}
}).ToListAsync();
I didn't need to include the Theme to make this work, e.g:
List<BookModel> books = await context.Books.Include(x => x.Theme).Select(x => ...
When will I need to use Include in Entity Framework?
UPDATE
I added a column of type Byte[] Illustration in Theme. In my projection I am not including that column so will it be loaded if I use Include? Or is never loaded unless I have it in the projection?
In search for an official answer to your question from Microsoft's side, I found this quote from Diego Vega (part of the Entity Framework and .NET team) made at the aspnet/Announcements github
repository:
A very common issue we see when looking at user LINQ queries is the use of Include() where it is unnecessary and cannot be honored. The typical pattern usually looks something like this:
var pids = context.Orders
.Include(o => o.Product)
.Where(o => o.Product.Name == "Baked Beans")
.Select(o =>o.ProductId)
.ToList();
One might assume that the Include operation here is required because of the reference to the Product navigation property in the Where and Select operations. However, in EF Core, these two things are orthogonal: Include controls which navigation properties are loaded in entities returned in the final results, and our LINQ translator can directly translate expressions involving navigation properties.
You didn't need Include because you were working inside EF context. When you reference Theme inside the anonymous object you are creating, that's not using lazy loading, that's telling EF to do a join.
If you return a list of books and you don't include the themes, then when you try to get the theme you'll notice that it's null. If the EF connection is open and you have lazy loading, it will go to the DB and grab it for you. But, if the connection is not opened, then you have to get it explicitely.
On the other hand, if you use Include, you get the data right away. Under the hood it's gonna do a JOIN to the necessary table and get the data right there.
You can check the SQL query that EF is generating for you and that's gonna make things clearer for you. You'll see only one SQL query.
If you Include a child, it is loaded as part of the original query, which makes it larger.
If you don't Include or reference the child in some other way in the query, the initial resultset is smaller, but each child you later reference will lazy load through a new request to the database.
If you loop through 1000 users in one request and then ask for their 10 photos each, you will make 1001 database requests if you don't Include the child...
Also, lazy loading requires the context hasn't been disposed. Always an unpleasant surprise when you pass an Entity to a view for UI rendering for example.
update
Try this for example and see it fail:
var book = await context.Books.First();
var theme = book.Theme;
Then try this:
var book = await context.Books.Include(b => b.Theme).First();
var theme = book.Theme;
I have a simple data model
car
- make
- model
- year
- colour
- engine
- model
- no. cylinders
- size
- etc
- fuel tank
- model
- capacity
- fuel type
- etc
- etc
So I have 'car', 'engine' and 'fuel tank' entities. Each of which have many properties.
I want a list of all the 100s of cars but only want to show the following selected properties: car.make, car.model, car.year, car.engine, car.size, car.fueltype.
I can certainly use .include to bring back sub-entities in the object graph but this is a big hit as there are many properties.
My question is whether there is a neat way to do this. Or any way in fact using Entity Framework (ideally EF7/Core)?
[ I did refer to https://colinmackay.scot/2011/07/31/getting-just-the-columns-you-want-from-entity-framework/ which uses the select into an anonymous class, but could not see how this could work within multiple includes ]
Thanks you.
You only need to use Include if you want to pull the full entities back - you don't need these to do a projection. You can do a projection either anonymously, or using a defined model class. The following code should get you started:
// Define model...
public class CarModel
{
public string Make { get; set; }
public string Model { get; set; }
public int EngineCC { get; set; }
}
// Project to list of models
var cars = context.Cars.Select(c => new CarModel
{
Make = c.Make,
Model = c.Model,
EngineCC = c.Engine.CC
}).ToList();
You can make this much simpler by using a mapping library such as AutoMapper. Using AutoMapper, this becomes:
// (at start of project)
Mapper.Initialize(c => {
c.CreateMap<Car, CarModel>();
});
// Projection...
var cars = context.Cars.ProjectTo<CarModel>().ToList();
In this example, EngineCC was automatically mapped from Engine.CC, but you can manually specify any mappings which don't just work automatically. AutoMapper will create a Linq projection, only bringing back the properties you need.
I have two relevant tables here:
public partial class List
{...
public int RegionId { get; set; }
[ForeignKey("RegionId")]
public virtual Region Region { get; set; }
...}
public partial class Region
{
public Region()
{
Lists = new HashSet<List>();
}
public int RegionId { get; set; }
[Required]
[StringLength(255)]
public string Name { get; set; }
public DateTime Added { get; set; }
public virtual ICollection<List> Lists { get; set; }
}
Here's the contents of the Regions table:
RegionId Name
1 Global
2 China
3 USA
4 UK
8 Canada
9 Spain
10 France
On the breeze side of things, I pull down the Regions:
var query = new breeze.EntityQuery().from("Regions").select("RegionId,Name").orderBy("RegionId");
return $rootScope.breezeManager.executeQuery(query).then(function (data) {
service.regions = data.results;
It seems to work:
service.regions[2]; // Name: "USA", RegionId: 3
However, when I try to create a new entity:
var newList = $rootScope.breezeManager.createEntity('List', listValues);
And in listValues, I specify { RegionId: 3, ...}:
newList.RegionId // 3
newList.Region // null
That would be strange already perhaps, but the really frustrating thing is, if I specify another value, like 1, it works. Same for 2, and 4:
newList.RegionId // 1
newList.Region.Name // "Global"
I've been poring through the code (and the internet) for hours trying to figure this out, but it's eluded me, and thus qualifies for my first ever SO question!
Update
I'm now even more confused. I planned to workaround this by manually setting the Region after the createEntity call, so I added this line of code right above createEntity:
var region = Enumerable.From(service.regions).Single("$.RegionId == " + listValues.RegionId);
However, after doing so, and with no other changes, newList now correctly gets a populated Region, even for 3 (USA)! I then commented out that line and tried again, and it still worked. Tried a few other times, various combinations of things, and now it's not working again, even with that line. Not sure if that helps. I'll keep experimenting.
I don't know what you're trying to do exactly but I do see that you have a fundamental misunderstanding.
I believe you are expecting your first query to return Region entities and you think that service.regions is populated with Region entities in the success callback.
Neither is true. Your query contains a select() clause which makes it what we call a projection. A projection returns data objects, not entities. There are no Region entities in cache either.
You may have seen elsewhere - perhaps a Pluralsight video - where a projection query did return entities. That happens when the query includes a toType clause which actually casts the projected data into instances of the targeted type. That's a trick you should only perform with great care, knowing what you are doing, why, and the limitations. But you're not casting in this query so this digression is besides the point.
It follows that the service.regions array holds some data objects that contain Region data but it does not hold Region entities.
This also explains why, when you created a new List entity with RegionId:3, the new entity's Region property returned null. Of course it is null. Based only on what you've told us, there are no Region entities in cache at all, let alone a Region with id=3.
I can't explain how you're able to get a Region with id=1 or id=2. I'm guessing there is something you haven't told us ... like you acquired these regions by way of some other query.
I don't understand why you're using a projection query in the first place. Why not just query for all regions and be done with it?
breeze.EntityQuery.from("Regions").orderBy("RegionId")
.using(manager).execute(function(data) {
service.regions = data.results;
});
I don't understand your update at all; that doesn't look like a valid EF query to me.
Tangential issues
First, why are you using a HashSet<List> to initialize the Lists property in your server-side Region model class? That's a specialized collection type that only confuses matters here. It doesn't do what the casual reader might think it does. While it prevents someone from adding the same object reference twice, it doesn't do the more important job of preventing someone from adding two different List objects with the same RegionId. The simple, obvious, and correct thing to do is ...
Lists = new System.Collections.Generic.List<List>();
// the full type name is only necessary because you burned the name "List"
// when defining your model class.
Second, on the client-side, please don't extend the Angular $rootScope with anything. That kind of global variable "pollution" is widely regarded as "bad practice". Keep your Breeze stuff inside a proper Angular service and inject that service when you need something.
I have a ServiceOperation to query items available to a certain user at a certain time.
In short this methode does:
var fullResult = from i in Items where ... select i; //get ALL possible items where...,
Lets say this returns items {A, B, C, D}.
A second query filters out which of those items the calling user has access to.
var clientResult = from ci in fullResult where (privilege's and schedule's are true)
This mite result in {A, C } and is returned.
The result on the client side is: only the list of items the client has access to is displayed. This can be annoying since you don't know if you made a mistake in searching, or the item is just not available right now.
What I would like to be able to do is show all possible results to the client {A, B, C, D} yet FLAG B and D in this case as unavailable.
My entity has already a property isReadOnly I could use.
Can I write a query to not just filter out, but also flag any remaining results as read only? An ideal result would be {A, B.isREadOnly=true, C, D.isReadOnly=true}
Or did I reach the limit of what is doable and do I have to write a traditional WCF web method, creating a separate class, returning a list of results?
PS: this 'isReadOnly' property is only used for this, I don't mind it being changed in the DB at all
Thanx for any pointers
Andreas
If I were you I would consider not returning the entity directly out of your service and instead map it to something that has the ReadOnly property. For example, if your entity is:
public class A
{
public string Name { get; set; }
}
Then you could have a data contract like this:
[DataContract]
public class AExtra
{
[DataMember]
public string Name { get; set; }
[DataMember]
public bool IsReadOnly { get; set; }
}
what this means is that you could do this in your query:
var res = from a
in Items
where (...whatever your original logic is)
select new AExtra
{
Name = a.Name,
IsReadOnly = (...put your logic for determining ReadOnly in here)
};
And then return res from your service operation.
Just an opinion really but I like to do things like this rather than send the entities directly out of the service - it always gives me a bit more freedom to change things without having too many knock-on effects.
I'm in the process of building the capability for a user to perform ad-hoc queries on a SQL Server database. The resulting query will take on the following basic form:
SELECT <ONE TO MANY USER SELECTED FIELDS>
FROM <ONE TO MANY TABLES DETERMINED BY FIELDS SELECTED BY USER>
WHERE <ZERO TO MANY CRITERIA FOR THE SELECTED FIELDS>
It's a guarantee that the selection will most likely span more than one table.
Some (not all) of the fields may have 0 or more filter criteria for a particular field.
My application is using the default EF4 classes within ASP.NET MVC 2 using C#. I am currently passing in an object called QueryItem that contains all the information for a particular criteria.
My question(s) are:
What is the best approach for coding this? (Code samples of what I have to date below).
Can this be done with Linq2SQL or should I use ADO.NET(My current approach)
If ADO.NET is the best way, how do you access the DBConnection within EF4?
Note: I intend to refactor this into SQLParameter objects, to protect against SQL injection. My goal right now is best practice in developing the query first.
QueryItem class:
public class QueryItem
{
public bool IsIncluded { get; set; }
public bool IsRequired { get; set; }
public string LabelText { get; set; }
public string DatabaseLoc { get; set; }
public List<string> SelectedValue { get; set; }
public List<SelectListItem> SelectList { get; set; }
}
Query Parsing Code
foreach(QueryItem qi in viewModel.StandardQueryItems)
{
string[] dLoc = qi.DatabaseLoc.Split(new Char[] { '.' }); //Split the table.fieldname value into a string array
if(qi.IsIncluded == true) //Check if the field is marked for inclusion in the final query
{
fields.Append(qi.DatabaseLoc + ","); //Append table.fieldname to SELECT statement
if(!tables.ToString().Contains(dLoc[0])) // Confirm that the table name has not already been added to the FROM statement
{
tables.Append(dLoc[0] + ","); //Append the table value to the FROM statement
}
}
if(qi.SelectedValue != null)
{
if(qi.SelectedValue.Count == 1)
{
query.Append(qi.DatabaseLoc + " = '" + qi.SelectedValue[0].ToString() + "'");
}
else
{
foreach(string s in qi.SelectedValue)
{
//Needs to handle "IN" case properly
query.Append(qi.DatabaseLoc + " IN " + qi.SelectedValue.ToString());
}
}
}
}
I have built a similar system to what you are describing in the past by passing in a single parameter to a stored procedure of type xml. By doing so, you can actually specify(in xml), what all you would like to report off of and build the SQL necessary to return the results you want.
This also makes your C# code easier, as all you have to do is generate some xml that your procedure will read. Generating Dynamic SQL is definitely not something you should use unless you have to, but when you want to allow users to dynamically select what they want to report off of, it's pretty much the only way to go about doing it.
Another option for you might be to look into Reporting Services - that will allow the user to pick what fields they want to view and save that particular 'report' in their own section where they can then go back and run it again at any time.. You could also create the reports for them if they aren't computer savvy(which is a lot easier to do with report builder, provided that all they need is data and no special features).
Either way you go about it, their are pros and cons to both solutions.. You'll have to determine which option is best for you.
xml/dynamic sql: Hard to maintain/make changes to.(I feel sorry for anyone who has to come behind someone who is generating dynamic sql and try to understand the logic behind the mess).
reporting services: very easy to spit out reports that look good, but it's a little less flexible and it's not free.