How to compare the price in EFCore with cosmos db.
Here is my products data
{
"name":"car",
"SalePrice" :
{
"price":"99.10",
"currency":"USD"
}
}
query:
query.Where(product => product.SalePrice.price > 1 and product.SalePrice.price < 100)
The issue here is the price in cosmos db is string, not decimal. How can I filter the products in SERVER side.
yes I can query.ToList() and then filter in client side, but this is not good solution if there is a lot of data, there could be out of memory issue if the result is big.
My requirement is howe can I write EFCore5.0 C# code to generate the below SQL API code for cosmos.
select * from Product p where StringToNumber(p.SalePrice.price) < 100 and StringToNumber(p.SalePrice.price) > 1;
in other words, how can I compare the string price as decimal price? Filter data in server database side, not in application server memory.
Using IQueryable for query data out-memory (Like server side, remote database, services) collections. IQueryable run query on server side and apply all filters on server then pass the data to client.
This query filter data on server side and then return after all filter apply on server side.
Code Example
public IActionResult Index()
{
List<Product> products = new List<Product>();
//getting data from Cosmosdb and filter on server side
IQueryable<Product> query = productlist().AsQueryable()
.Where(s => Convert.ToDecimal(s.Price) > 100
&& Convert.ToDecimal(s.Price) < 250);
return View();
}
More information about IQueryable by Microsoft Document.
Related
I'm trying to optimize my EF queries. I have an entity called Employee. Each employee has a list of tools. Ultimately, I'm trying to get a list of employees with their tools that are NOT broken. When running my query, I can see that TWO calls are made to the server: one for the employee entities and one for the tool list. Again, I'm trying to optimize the query, so the server is hit for a query only once. How can I do this?
I've been exploring with LINQ's join and how to create a LEFT JOIN, but the query is still not optimized.
In my first code block here, the result is what I want, but -- again -- there are two hits to the server.
public class Employee
{
public int EmployeeId { get; set; }
public List<Tool> Tools { get; set; } = new List<Tool>();
...
}
public class Tool
{
public int ToolId { get; set; }
public bool IsBroken { get; set; } = false;
public Employee Employee { get; set; }
public int EmployeeId { get; set; }
...
}
var x = (from e in db.Employees.Include(e => e.Tools)
select new Employee()
{
EmployeeId = e.EmployeeId,
Tools = e.Tools.Where(t => !t.IsBroken).ToList()
}).ToList();
This second code block pseudoly mimics what I'm trying to accomplish. However, the GroupBy(...) is being evaluated locally on the client machine.
(from e in db.Employees
join t in db.Tools.GroupBy(tool => tool.EmployeeId) on e.EmployeeId equals t.Key into empTool
from et in empTool.DefaultIfEmpty()
select new Employee()
{
EmployeeId = e.EmployeeId,
Tools = et != null ? et.Where(t => !t.IsBroken).ToList() : null
}).ToList();
Is there anyway that I can make ONE call to the server as well as not having my GroupBy() evaluate locally and have it return a list of employees with a filtered tool list with tools that are not broken? Thank you.
Shortly, it's not possible (and I don't think it ever will be).
If you really want to control the exact server calls, EF Core is simply not for you. While EF Core still has issues with some LINQ query translation which leads to N+1 query or client evaluation, one thing is by design: unlike EF6 which uses single huge union SQL query for producing the result, EF Core uses one SQL query for the main result set plus one SQL query per each correlated result set.
This is sort of explained in the How Queries Work EF Core documentation section:
The LINQ query is processed by Entity Framework Core to build a representation that is ready to be processed by the database provider
The result is cached so that this processing does not need to be done every time the query is executed
The result is passed to the database provider
The database provider identifies which parts of the query can be evaluated in the database
These parts of the query are translated to database specific query language (for example, SQL for a relational database)
One or more queries are sent to the database and the result set returned (results are values from the database, not entity instances)
Note the word more in the last bullet.
In your case, you have 1 main result set (Employee) + 1 correlated result set (Tool), hence the expected server queries are TWO (except if the first query returns empty set).
You can use this:
var x = from e in _context.Employees
select new
{
e,
Tools = from tool in e.Tools where !tool.IsBroken select tool
};
var result = x.AsEnumerable().Select(y => y.e);
Which will be finally translated to a SQL query like below depending on your provider:
SELECT
`Project1`.`EmployeeId`,
`Project1`.`Name`,
`Project1`.`C1`,
`Project1`.`ToolId`,
`Project1`.`IsBroken`,
`Project1`.`EmployeeId1`
FROM (SELECT
`Extent1`.`EmployeeId`,
`Extent1`.`Name`,
`Extent2`.`ToolId`,
`Extent2`.`IsBroken`,
`Extent2`.`EmployeeId` AS `EmployeeId1`,
CASE WHEN (`Extent2`.`ToolId` IS NOT NULL) THEN (1) ELSE (NULL) END AS `C1`
FROM `Employees` AS `Extent1` LEFT OUTER JOIN `Tools` AS `Extent2` ON (`Extent1`.`EmployeeId` = `Extent2`.`EmployeeId`) AND (`Extent2`.`IsBroken` != 1)) AS `Project1`
ORDER BY
`Project1`.`EmployeeId` ASC,
`Project1`.`C1` ASC
I change my previous answer which was wrong, thanks to comments.
I'm using entity framework to connect to database from my application. I have table in SQL, named Orders. It contains such fields as: TransactionId, ParticipantId and is linked to Transactions table which has one to many connection to Participants table. I need to get data from it using List of classes with such properties: TransactionId, ParticipantId, OrganizationId. Linq must meet such conditions: (orders.TransactionId == TransactionId && orders.ParticipantId == ParticipantId && orders.Transaction.Participants.Any(x=> x.Id == OrganizationId)). This should be done by one query, not by multiple, so, please don't recommend foreach or smth like that.
Like #NetMage said, generally we need examples. Assuming that you've got a dbcontext set up, the ask is pretty simple:
public static void GetData(int transactionId, int participantId, int organizationId)
{
using (var db = new MyDbContext())
{
var query =
(
from t in db.Transactions
from o in db.Orders
.Where(w => w.TransactionId == t.TransactionId)
from p in db.Participants
.Where(w => w.TransactionId == t.TransactionId)
where t.TransactionId = transactionId &&
o.ParticipantId = participantId
select new { Order = o, Transaction = t, Participant = p}
);
}
}
Again since we don't have a lot of information here it's hard to do more. You should be able to take it from there. I know I didn't use the organizationId filter, but since I don't know the target shape of the data I'm not sure what the best path would be
Environment : MongoDb 3.2, Morphia 1.1.0
So lets say i am having a collection of Employees and Employee entity has several fields. I need to do something like apply multiple filters (conditional) and return a batch of 10 records per request.
pesudocode as below.
#Entity("Employee")
Employee{
String firstname,
String lastName,
int salary,
int deptCode,
String nationality
}
and in my EmployeeFilterRequesti carry the request parameter to the dao
EmployeeFilterRequest{
int salaryLessThen
int deptCode,
String nationality..
}
Pseudoclass
class EmployeeDao{
public List<Employee> returnList;
public getFilteredResponse(EmployeeFilterRequest request){
DataStore ds = getTheDatastore();
Query<Employee> query = ds.createQuery(Emploee.class).disableValidation();
//conditional request #1
if(request.filterBySalary){
query.filter("salary >", request.salary);
}
//conditional request #2
if(request.filterBydeptCode){
query.filter("deptCode ==", request.deptCode);
}
//conditional request #3
if(request.filterByNationality){
query.filter("nationality ==", request.nationality);
}
returnList = query.batchSize(10).asList();
/******* **THIS IS RETURNING ME ALL THE RECORDS IN THE COLLECTION, EXPECTED ONLY 10** *****/
}
}
SO as explained above in the code.. i want to perform conditional filtering on multiple fields. and even if batchSize is present as 10, i am getting complete records in the collection.
how to resolve this ???
Regards
Punith
Blakes is right. You want to use limit() rather than batchSize(). The batch size only affects how many documents each trip to the server comes back with. This can be useful when pulling over a lot of really large documents but it doesn't affect the total number of documents fetched by the query.
As a side note, you should be careful using asList() as it will create objects out of every document returned by the query and could exhaust your VM's heap. Using fetch() will let you incrementally hydrate documents as you need each one. You might actually need them all as a List and with a size of 10 this is probably fine. It's just something to keep in mind as you work with other queries.
I would like to know what happens when I use IQueryable with and without AsQueryable(). Here is an example:
public partial class Book
{
.......
public Nullable<System.DateTime> CheckoutDate{get; set;}
}
I need to filter the data from SQL server before it is returned to an application server. I need to return books checked out more recently than entered date. Which one should I use?
A.
IQueryable<Book> books = db.Books;
books = books.Where(b => b.CheckoutDate >= date);
B.
IQueryable<Book> books = db.Books.ToList().AsQueryable();
books = books.Where(b => b.CheckoutDate >= date);
Basically I would like to know what is the difference between the above two options. Do they work on the similar grounds? Do they return same values?
With B option, you're basically retrieving every book from database and filtering data in memory.
A option is more performance, as it filters data at the database and return only the rows that match your query.
I'm developing a client app that uses breezejs and Entity Framework 6 on the back end. I've got a statement like this:
var country = 'Mexico';
var customers = EntityQuery.from('customers')
.where('country', '==', country)
.expand('order')
I want to use There may be hundreds of orders that each customer has made. For the purposes of performance, I only want to retrieve the latest order for each customer. This will be based on the created date for the order. In SQL, I could write something like this:
SELECT c.customerId, companyName, ContactName, City, Country, max(o.OrderDate) as LatestOrder FROM Customers c
inner join Orders o on c.CustomerID = o.CustomerID
group by c.customerId, companyName, ContactName, City, Country
If this was run against the northwind database, only the most recent order row is returned for each customer.
How can I write a similar query in breeze, so that it runs on the server side and therefore returns less data to the client. I know I could handle this all on the client but writing some javascript in a querysucceeded method that could be run by the client - but that's not the goal here.
thanks
For a case like this, you should create a special endpoint method that will perform your query.
Then you can use an Entity Framework query to do what you want, using the LINQ syntax.
Here are two Web API examples:
[HttpGet]
public IQueryable<Object> CustomersLatestOrderEntities()
{
// IQueryable<Object> containing Customer and Order entity
var entities = ContextProvider.Context.Customers.Select(c => new { Customer = c, LatestOrder = c.Orders.OrderByDescending(o => o.OrderDate).FirstOrDefault() });
return entities;
}
[HttpGet]
public IQueryable<Object> CustomersLatestOrderProjections()
{
// IQueryable<Object> containing Customer and Order entity
var entities = ContextProvider.Context.Customers.Select(c => new { Customer = c, LatestOrder = c.Orders.OrderByDescending(o => o.OrderDate).FirstOrDefault() });
// IQueryable<Object> containing just data fields, no entities
var projections = entities.Select(e => new { e.Customer.CustomerID, e.Customer.ContactName, e.LatestOrder.OrderDate });
return projections;
}
Note that you have a choice here. You can return actual entities, or you can return just some data fields. Which is right for you depends upon how you are going to use them on the client. If they are just for display in a
non-editable list, you can just return the plain data (CustomersLatestOrderProjections above). If they can potentially
be edited, then return the object containing the entities (CustomersLatestOrderEntities). Breeze will merge the entities
into its cache, even though they are contained inside this anonymous object.
Either way, because it returns IQueryable, you can use the Breeze filtering syntax from the client to further qualify the query.
var projectionQuery = breeze.EntityQuery.from("CustomersLatestOrderProjections")
.skip(20)
.take(10);
var entityQuery = breeze.EntityQuery.from("CustomersLatestOrderEntities")
.where('customer.countryName', 'startsWith', 'C');
.take(10);