Dynamic Search Using Entity Framework - entity-framework

I have a search screen with optional fields using Entity Framework, I want to build a dynamic query without selecting the object and filter on it.
I want to Optimize the following Existing Search, I don't want to select "var query = from p in context.APLCN_TRCKR select p;" at this stage because the application will be used by more than 100 people at once:
using (var context = new VASTEntities())
{
var query = from p in context.APLCN_TRCKR select p;
if (!string.IsNullOrEmpty(searchObj.CUST_NAME_X))
query = query.Where(p => p.CUST_NAME_X == searchObj.CUST_NAME_X.Trim());
if (!string.IsNullOrEmpty(searchObj.SURNM_X))
query = query.Where(p => p.CUST_SURNM_X == searchObj.SURNM_X.Trim());
if (!string.IsNullOrEmpty(searchObj.QUEUE_ID))
query = query.Where(p => p.QUEUE_ID == searchObj.QUEUE_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.BP_ID))
query = query.Where(p => p.BPID == searchObj.BP_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.UserID))
query = query.Where(p => p.CURR_OWNR_USER_ID == searchObj.UserID.Trim());
if (!string.IsNullOrEmpty(searchObj.APLCN_TRCKR_ID))
query = query.Where(p => p.APLCN_TRCKR_ID == searchObj.APLCN_TRCKR_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.APLCN_STTS_ID))
query = query.Where(p => p.APLCN_STTS_ID == searchObj.APLCN_STTS_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.CUST_ID))
query = query.Where(p => p.CUST_ID == searchObj.CUST_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.CELL_ID))
query = query.Where(p => p.CELL_ID == searchObj.CELL_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.ORIGN_ID))
query = query.Where(p => p.ORIGN_ID == searchObj.ORIGN_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.ORGTN_CHANL_ID))
query = query.Where(p => p.ORGTN_CHANL_ID == searchObj.ORGTN_CHANL_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.CR_DCSN_ID))
query = query.Where(p => p.CR_DCSN_ID == searchObj.CR_DCSN_ID.Trim());
if (!string.IsNullOrEmpty(searchObj.SBSA_CUST_I))
query = query.Where(p => p.SBSA_CUST_I == searchObj.SBSA_CUST_I.Trim());
if (!string.IsNullOrEmpty(searchObj.USER_ID_APP_CRTD))
query = query.Where(p => p.USER_ID_APP_CRTD == searchObj.USER_ID_APP_CRTD.Trim());
if (!string.IsNullOrEmpty(searchObj.RGION_ID))
{
int r = int.Parse(searchObj.RGION_ID.Trim());
query = query.Where(p => p.RGION_ID == r);
}
if (!string.IsNullOrEmpty(searchObj.CR_REGION))
{
int x = int.Parse(searchObj.CR_REGION);
if (x == 0)
{
// check 0 - not applicable or null
query = query.Where(p => p.CR_REGION_ID == 0 || p.CR_REGION_ID == null);
}
else
{
query = query.Where(p => p.CR_REGION_ID == x);
}
}
if (!string.IsNullOrEmpty(searchObj.Process_Type))
query = query.Where(p => p.PRCES_TYPE_ID == searchObj.Process_Type.Trim());
query.ToList();
foreach (var a in query)
{
searchAppsObj.Add(Translator.TranslateReqObjToBO.TranslateDTOToSearchApp(a));
}
if (query.Count() == 0)
{
throw new Exception("No Applications Found.");
}
context.Connection.Close();
return searchAppsObj;
}
I want to do something like this but this one is not working properly:
string cust_name_x = "", surname_x = "", queue_id = "", bp_id = "", user_id = "", aplcn_trckr_id = "",
aplcn_stts_id = "", cust_id = "", process_type = "", cr_region = "", cell_id = "", Origin = "", region = "", channel = "", credit_verdict = "", sbsa_cust_id = "", app_creator_id = "";
if (!string.IsNullOrEmpty(searchObj.CUST_NAME_X))
cust_name_x = searchObj.CUST_NAME_X.Trim();
if (!string.IsNullOrEmpty(searchObj.SURNM_X))
surname_x = searchObj.SURNM_X.Trim();
if (!string.IsNullOrEmpty(searchObj.QUEUE_ID))
queue_id = searchObj.QUEUE_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.BP_ID))
bp_id = searchObj.BP_ID;
if (!string.IsNullOrEmpty(searchObj.UserID))
user_id = searchObj.UserID.Trim();
if (!string.IsNullOrEmpty(searchObj.APLCN_TRCKR_ID))
aplcn_trckr_id = searchObj.APLCN_TRCKR_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.APLCN_STTS_ID))
aplcn_stts_id = searchObj.APLCN_STTS_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.CUST_ID))
cust_id = searchObj.CUST_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.Process_Type))
process_type = searchObj.Process_Type.Trim();
if (!string.IsNullOrEmpty(searchObj.CR_REGION))
cr_region = searchObj.CR_REGION.Trim();
if (!string.IsNullOrEmpty(searchObj.CELL_ID))
cell_id = searchObj.CELL_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.ORIGN_ID))
Origin = searchObj.ORIGN_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.RGION_ID))
region = searchObj.RGION_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.ORGTN_CHANL_ID))
channel = searchObj.ORGTN_CHANL_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.CR_DCSN_ID))
credit_verdict = searchObj.CR_DCSN_ID.Trim();
if (!string.IsNullOrEmpty(searchObj.SBSA_CUST_I))
sbsa_cust_id = searchObj.SBSA_CUST_I.Trim();
if (!string.IsNullOrEmpty(searchObj.USER_ID_APP_CRTD))
app_creator_id = searchObj.USER_ID_APP_CRTD.Trim();
using (var context = new VASTEntities())
{
var query = from p in context.APLCN_TRCKR
where
p.CUST_NAME_X.Contains(cust_name_x) &&
p.CUST_SURNM_X.Contains(surname_x) &&
p.QUEUE_ID.Contains(queue_id) &&
p.BPID.Contains(bp_id) &&
p.CURR_OWNR_USER_ID.Contains(user_id) &&
p.APLCN_TRCKR_ID.Contains(aplcn_trckr_id) &&
p.APLCN_STTS_ID.Contains(aplcn_stts_id) &&
p.CUST_ID.Contains(cust_id) &&
p.PRCES_TYPE_ID.Contains(process_type) &&
p.CELL_ID.Contains(cell_id) &&
SqlFunctions.StringConvert((double)p.CR_REGION_ID).Contains(cr_region) &&
p.ORIGN_ID.Contains(Origin) &&
SqlFunctions.StringConvert((double)p.RGION_ID).Contains(region) &&
p.ORGTN_CHANL_ID.Contains(channel) &&
p.CR_DCSN_ID.Contains(credit_verdict) &&
p.SBSA_CUST_I.Contains(sbsa_cust_id)
select p;
query.ToList();
if (query.Count() == 0)
{
throw new Exception("No Applications Found.");
}
foreach (var a in query)
{
searchAppsObj.Add(Translator.TranslateReqObjToBO.TranslateDTOToSearchApp(a));
}
context.Connection.Close();
return searchAppsObj;
}

You can just create a collection of lambda expression like below:
var filters = new List<Expression<Func<Application, bool>>>();
if (!string.IsNullOrWhitespace(searchObj.CUST_NAME_X))
filters.Add(application => application .CUST_NAME_X.Contains(searchObj.CUST_NAME_X.Trim());
if (!string.IsNullOrEmpty(searchObj.SURNM_X))
filters.Add(application => application .CUST_SURNM_X.Contains(searchObj.SURNM_X.Trim());
// And so on for all criteria
After that you can do a loop on filters like below:
using (var context = new VASTEntities())
{
var query = context.APLCN_TRCKR;
foreach(var filter in filters)
{
query = query.Where(filter);
}
var result = query.ToList();
if (result.Count() == 0)
{
throw new Exception("No Applications Found.");
}
foreach (var a in result)
{
searchAppsObj.Add(Translator.TranslateReqObjToBO.TranslateDTOToSearchApp(a));
}
context.Connection.Close();
return searchAppsObj;
}

change
var query = from p in context.APLCN_TRCKR select p;
to
var query = context.APLCN_TRCKR.AsQueryable();
And when the filtering work is done:
await query.ToListAsync() // this one goes to the database
Also have a look at this: Linq query filtering
In Addition: don't call context.Connection.Close(); as this is going to be executed anyway because using behaves like try { ... } finally { // dispose work }

Related

ASP.NET Core MVC : GET details using view model not working

I am having a problem using a view model to view get the details of an item selected in the view. I am loading a result into my index view as follows
public async Task<IActionResult> Index(string sortOrder, string currentFilter, string searchString, int? page)
{
ViewBag.CurrentSort = sortOrder;
ViewBag.ConOrgSortParm = String.IsNullOrEmpty(sortOrder) ? "Client_desc" : "Client";
ViewBag.AssignedSortParm = String.IsNullOrEmpty(sortOrder) ? "Assigned_desc" : "Assigned";
ViewBag.ExpiresSortParm = String.IsNullOrEmpty(sortOrder) ? "Expires_desc" : "Expires";
ViewBag.LastActiveSortParm = String.IsNullOrEmpty(sortOrder) ? "LastActive_desc" : "LastActive";
ViewBag.IDSortParm = String.IsNullOrEmpty(sortOrder) ? "Id_desc" : "";
if (searchString != null)
{
page = 1;
}
else
{
searchString = currentFilter;
}
ViewBag.CurrentFilter = searchString;
var results = (from node in _context.Nodes
join bnbridge in _context.BundleNodes on node.Id equals bnbridge.NodeId into NodeBundleIDGroup
from ax in NodeBundleIDGroup.DefaultIfEmpty()
join bundle in _context.Bundles on ax.BundleId equals bundle.Id into NodeBundleGroup
from bx in NodeBundleGroup.DefaultIfEmpty()
join agreement in _context.Agreements on bx.AgreementId equals agreement.Id into agGroup
from cx in agGroup.DefaultIfEmpty()
join org in _context.Organizations on cx.OrgId equals org.Id into oGroup
from ex in oGroup.DefaultIfEmpty()
join conorg in _context.Organizations on node.OrgId equals conorg.Id into tGroup
from fx in tGroup.DefaultIfEmpty()
select new NodeIndexViewModel
{
Id = node.Id,
Name = node.Name,
AssignedOrg = fx.ShortName,
ContractingOrg = ex.ShortName,
Expiry = bx.EndUtc,
LastActive = node.ActiveDate,
NodeType = bx.NodeTypeID
});
if (!String.IsNullOrEmpty(searchString))
{
results = results.Where(s => s.AssignedOrg.Contains(searchString)
|| s.Id.ToString().StartsWith(searchString));
}
switch (sortOrder)
{
case "Client_desc":
results = results.OrderByDescending(s => s.ContractingOrg);
break;
case "Client":
results = results.OrderBy(s => s.ContractingOrg);
break;
case "Assigned":
results = results.OrderBy(s => s.AssignedOrg);
break;
case "Assigned_desc":
results = results.OrderByDescending(s => s.AssignedOrg);
break;
case "Expires":
results = results.OrderBy(s => s.Expiry);
break;
case "Expires_desc":
results = results.OrderByDescending(s => s.Expiry);
break;
case "LastActive":
results = results.OrderBy(s => s.LastActive);
break;
case "LastActive_desc":
results = results.OrderByDescending(s => s.LastActive);
break;
case "Id_desc":
results = results.OrderByDescending(s => s.Id);
break;
default:
results = results.OrderBy(s => s.Id);
break;
};
int pageSize = 10;
int pageNumber = (page ?? 1);
return View(await results.ToPagedListAsync(pageNumber, pageSize));
}
This is working as expected. The problem comes when I want to access the Details of one of the listed results.
I have tried the following in the controller:
public async Task<IActionResult> Details(int? id)
{
if (id == null || _context.NodeIndexViewModel == null)
{
return NotFound();
}
var nodeIndexViewModel = await _context.NodeIndexViewModel
.FirstOrDefaultAsync(m => m.Id == id);
if (nodeIndexViewModel == null)
{
return NotFound();
}
return View(nodeIndexViewModel);
}
I get an error
PostgresException: 42P01: relation "NodeIndexViewModel" does not exist
Can't answer the question because I don't know what the problem was but I rescaffolded the database and it's working now.

How to do both Add and Update in Entity Framework ASP.NET Core?

I want to do both updates and add in the same action method in ASP.NET Core by Entity Framework Core, but after making an addition to a list of records in the database, I cannot update the same records to the table. It always creates a new set of records.
What is my mistake? Can anybody please help me?
[HttpPost]
public IActionResult InsertProductDetails()
{
using WebClient wc = new WebClient();
string contentString = wc.DownloadString(baseurl);
List<Dictionary<string, string>> ListJsonProductContent = new List<Dictionary<string, string>>();
var token = JToken.Parse(contentString);
if (token.Type == JTokenType.Array) // "["
{
ListJsonProductContent = JsonConvert.DeserializeObject<List<Dictionary<string, string>>>(contentString);
}
else if (token.Type == JTokenType.Object) // "{"
{
var ObjectResponse = JsonConvert.DeserializeObject<Dictionary<string, object>>(contentString);
foreach (var x in ObjectResponse)
{
string key = x.Key.ToString();
string val = x.Value.ToString();
foreach (var dicItemML in JsonConvert.DeserializeObject<List<Dictionary<string, string>>>(val))
{
ListJsonProductContent.Add(dicItemML);
}
}
}
List<K360MappingMaster> ListMappedDataDb = new List<K360MappingMaster>();
var VLinqQuery = from KMM in _context.K360MappingMasters
where KMM.ApiUrl != null && KMM.ApiUrl == baseurl
select KMM;
ListMappedDataDb = VLinqQuery.ToList();
foreach (var dicItemML in ListJsonProductContent)
{
Dictionary<string, string> updItem = new Dictionary<string, string>();
foreach (var itemMl in dicItemML)
{
if (ListMappedDataDb.Select(s => s.ApiCatalog).ToList().Contains(itemMl.Key))
{
if (updItem.ContainsKey(ListMappedDataDb.Where(s => s.ApiCatalog == itemMl.Key).Select(s => s.K360Catalog).FirstOrDefault()))
{
if (ListMappedDataDb.Where(s => s.ApiCatalog == itemMl.Key).Select(s => s.K360Catalog).FirstOrDefault() == "Specification")
{
updItem[ListMappedDataDb.Where(s => s.ApiCatalog == itemMl.Key).Select(s => s.K360Catalog).FirstOrDefault()] += "<p>" + itemMl.Key + " :" + itemMl.Value + "<p>";
}
else
{
updItem[ListMappedDataDb.Where(s => s.ApiCatalog == itemMl.Key).Select(s => s.K360Catalog).FirstOrDefault()] += " " + itemMl.Value;
}
}
else
{
if (ListMappedDataDb.Where(s => s.ApiCatalog == itemMl.Key).Select(s => s.K360Catalog).FirstOrDefault() == "Specification")
{
updItem.Add(ListMappedDataDb.Where(s => s.ApiCatalog == itemMl.Key).Select(s => s.K360Catalog).FirstOrDefault(), "<p>" + itemMl.Key + " :" + itemMl.Value + "<p>");
}
else
{
updItem.Add(ListMappedDataDb.Where(s => s.ApiCatalog == itemMl.Key).Select(s => s.K360Catalog).FirstOrDefault(), itemMl.Value);
}
}
}
dicItemML.Remove(itemMl.Key);
}
foreach (var itemM2 in updItem)
{
dicItemML.Add(itemM2.Key, itemM2.Value);
}
}
List<CatalogProduct> ListKp = new List<CatalogProduct>();
foreach (var dicItem in ListJsonProductContent)
{
try
{
CatalogProduct Ctgkp = new CatalogProduct
{
Name = dicItem.ContainsKey("Name") ? dicItem["Name"] : "No Product",
Slug = dicItem.ContainsKey("Name") ? string.Concat(dicItem["Name"].Where(c => !char.IsWhiteSpace(c))).ToLower() : "No Slug",
Price = dicItem.ContainsKey("Price") ? decimal.Parse(dicItem["Price"], CultureInfo.InvariantCulture) : default,
ShortDescription = dicItem.ContainsKey("ShortDescription") ? dicItem["ShortDescription"] : null,
Description = dicItem.ContainsKey("Description") ? dicItem["Description"] : null,
Specification = dicItem.ContainsKey("Specification") ? dicItem["Specification"] : null,
RatingAverage = dicItem.ContainsKey("RatingAverage") ? double.Parse(dicItem["RatingAverage"], CultureInfo.InvariantCulture) : null,
MetaTitle = dicItem.ContainsKey("MetaTitle") ? dicItem["MetaTitle"] : null,
MetaKeywords = dicItem.ContainsKey("MetaKeywords") ? dicItem["MetaKeywords"] : null,
MetaDescription = dicItem.ContainsKey("MetaDescription") ? dicItem["MetaDescription"] : null,
Sku = dicItem.ContainsKey("Sku") ? dicItem["Sku"] : null,
Gtin = dicItem.ContainsKey("Gtin") ? dicItem["Gtin"] : null,
NormalizedName = dicItem.ContainsKey("NormalizedName") ? dicItem["NormalizedName"] : null,
StockQuantity = dicItem.ContainsKey("StockQuantity") ? int.Parse(dicItem["StockQuantity"], CultureInfo.InvariantCulture) : 50,
ReviewsCount = dicItem.ContainsKey("ReviewsCount") ? int.Parse(dicItem["ReviewsCount"], CultureInfo.InvariantCulture) : default,
DisplayOrder = dicItem.ContainsKey("DisplayOrder") ? int.Parse(dicItem["DisplayOrder"], CultureInfo.InvariantCulture) : 1,
OldPrice = dicItem.ContainsKey("OldPrice") ? decimal.Parse(dicItem["OldPrice"], CultureInfo.InvariantCulture) : null,
SpecialPrice = dicItem.ContainsKey("SpecialPrice") ? decimal.Parse(dicItem["SpecialPrice"], CultureInfo.InvariantCulture) : null,
SpecialPriceStart = dicItem.ContainsKey("SpecialPriceStart") ? DateTimeOffset.Parse(dicItem["SpecialPriceStart"], CultureInfo.InvariantCulture) : null,
SpecialPriceEnd = dicItem.ContainsKey("SpecialPriceEnd") ? DateTimeOffset.Parse(dicItem["SpecialPriceEnd"], CultureInfo.InvariantCulture) : null,
IsPublished = true,
PublishedOn = DateTimeOffset.Now,
CreatedById = 10,
IsDeleted = false,
CreatedOn = DateTimeOffset.UtcNow,
LatestUpdatedOn = DateTimeOffset.UtcNow,
LatestUpdatedById = 10,
HasOptions = false,
IsVisibleIndividually = true,
IsFeatured = true,
IsCallForPricing = false,
IsAllowToOrder = true,
StockTrackingIsEnabled = true
};
ListKp.Add(Ctgkp);
}
catch (Exception ex)
{
TempData["Exceptionmsg"] = ex;
return RedirectToAction("Index");
throw;
}
}
int numP = 0;
try
{
using (var transaction = _context.Database.BeginTransaction())
{
int getcount = (from gcM in _context.CatalogProducts
select gcM).Count();
if (getcount == 0)
{
_context.CatalogProducts.AddRange(ListKp);
_context.SaveChanges();
}
else
{
var catalogProducts = (from gcM in _context.CatalogProducts select gcM).ToList();
for (int i = 0; i < catalogProducts.Count; i++)
{
catalogProducts[i].Name = ListKp[i].Name;
catalogProducts[i].Price = ListKp[i].Price;
catalogProducts[i].Description = ListKp[i].Description;
catalogProducts[i].Specification = ListKp[i].Specification;
//...
}
_context.CatalogProducts.UpdateRange(catalogProducts);
_context.SaveChanges();
}
transaction.Commit();
}
if (numP > 0)
{
TempData["Sucessmsg"] = "No conflicts. " + numP + " product details saved.";
(from p in _context.K360MappingMasters
where p.ApiUrl == baseurl
select p).ToList()
.ForEach(x => x.InsertStatusFlag = true);
_context.SaveChanges();
return RedirectToAction("Index");
}
}
catch (Exception ex)
{
TempData["Exceptionmsg"] = ex;
return RedirectToAction("Index"); ;
throw;
}
return RedirectToAction("Index");
}
I have accepted your answer. Thanks. But I have few more clarifications need for the same question
1.CatalogProduct table has Auto increment primary key Id field only. I am inserting data from JSON URL. The unique field should be the product name. How to check whether the product name already exists in the table before adding/updating records.
2.If I need to add another JSON URL data, how do I differentiate the two URL product details?
There is no entities in db being tracked when doing Update, so it will directly add them.
You can try the below codes.
else
{
var catalogProducts= (from gcM in _context.CatalogProducts select gcM).ToList();
for (int i = 0; i < catalogProducts.Count; i++)
{
catalogProducts[i].Name = ListKp[i].Name;
catalogProducts[i].Price = ListKp[i].Price;
//...
}
_context.CatalogProducts.UpdateRange(catalogProducts);
numP = _context.SaveChanges();
}
Update:
using (var transaction = _context.Database.BeginTransaction())
{
var catalogProducts = _context.CatalogProducts.ToList();
foreach(var kp in ListKp)
{
if(!catalogProducts.Any(x => x.Name == kp.Name)){
_context.CatalogProducts.Add(kp);
_context.SaveChanges();
}
else
{
//Use AutoMapper automatically do the mapping
var config = new MapperConfiguration(cfg => cfg.CreateMap<CatalogProduct, CatalogProduct>().ForMember(c => c.Id, opt => opt.Ignore()));
var oldone = catalogProducts.FirstOrDefault(c => c.Name == kp.Name);
var mapper = config.CreateMapper();
oldone = mapper.Map<CatalogProduct,CatalogProduct>(kp, oldone);
_context.CatalogProducts.Update(oldone);
_context.SaveChanges();
}
}
transaction.Commit();
}

Upgraded to .NET Core 3.1 and receiving an error in a LINQ query related to using .FirstOrDefault()

I have an Entity Framework controller that has been using the method below successfully. However I recently updated my project to use .NET Core 3.1 and it must've broken something.
I am now getting this error:
FirstOrDefault()' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()
I did some research, and some people say not to use GroupBy extension which I do in the query below. I tried to take it out, but that just generates more errors.
I also went to this:
But I couldn't figure out how to fix my complex query below.
Honestly, I'm not sure why it's failing or how to properly fix it.
Does anyone see anything wrong?
Thanks!
public async Task<ActionResult<object>> GetStarChemicalData(string starID)
{
var starChemicalData = await (from starlist in _context.StarList
join ql in _context.ChemicalList on starlist.ChemicalId equals ql.ChemicalId into stars
from chemicallist in stars.DefaultIfEmpty()
join qc in _context.ChemicalAtoms on chemicallist.ChemicalId equals qc.ChemicalId into chemicals
from chemicalatoms in chemicals.DefaultIfEmpty()
join nk in _context.StarLinks on chemicalatoms.AtomId equals nk.AtomId into links
from starlinks in links.DefaultIfEmpty()
where starlist.StarId == starID
select new
{
StarId = starlist.StarId,
StarType = starlist.StarType,
StarTitle = starlist.StarTitle,
ChemicalId = starlist.ChemicalId,
AtomId = (Guid?)chemicalatoms.AtomId,
OrderId = chemicalatoms.OrderId,
ChemicalText = chemicallist.ChemicalText,
AtomText = chemicalatoms.AtomText,
Wavelength = chemicalatoms.Wavelength,
isRedShifted = (starlinks.AtomId != null && starlist.StarType == 1) ? 1
: (starlinks.AtomId == null && starlist.StarType == 1) ? 0
: (int?)null
})
.GroupBy(x => x.StarId)
.Select(g => new
{
StarId = g.FirstOrDefault().StarId,
StarType = g.FirstOrDefault().StarType,
StarTitle = g.FirstOrDefault().StarTitle,
ChemicalId = g.FirstOrDefault().ChemicalId,
ChemicalText = g.FirstOrDefault().ChemicalText,
ChemicalAtoms = (g.FirstOrDefault().AtomId != null ? g.Select(x => new
{
AtomId = x.AtomId,
OrderId = x.OrderId,
AtomText = x.AtomText,
Feedback = x.Wavelength,
IsCorrect = x.isRedShifted
}) : null)
}).FirstOrDefaultAsync();
return starChemicalData;
ERROR AFTER DEBUGGING:
.Select(x => new {
AtomId = x.AtomId,
OrderId = x.OrderId,
AtomText = x.AtomText,
Feedback = x.Wavelength,
IsCorrect = x.isRedShifted
})' could not be translated.
You have alot of FirstOrDefault (s) calls.
To code them "safely", you can try this:
.FirstOrDefault() ?? string.Empty
or
.FirstOrDefault() ?? 0
..
the above is a short hand version of this: (null check + ? ternary operator) (again, this is a "safe" way to code it)
StarId = null == g.FirstOrDefault() ? 0 : g.FirstOrDefault().StarId,
StarType = null == g.FirstOrDefault() ? string.Empty : g.FirstOrDefault().StarType,
StarTitle = null == g.FirstOrDefault() ? string.Empty : g.FirstOrDefault().StarTitle,
..
But better to debug, to figure out what is wrong
Change ALL OF these (temporarilty) to "" and 0
I'm listing 3 of them, but you should change all of them
instead of this:
StarId = g.FirstOrDefault().StarId,
StarType = g.FirstOrDefault().StarType,
StarTitle = g.FirstOrDefault().StarTitle,
use (temporarily) this:
StarId = 0,
StarType = "",
StarTitle = "",
(again do ALL of them)
and one-by-one, replace them back
StarId = g.FirstOrDefault().StarId,
StarType = "",
StarTitle = "",
to find the "culprit".
Here is some pseudo code.......that you can try to use the intermediate step of IQueryable. Its pseduo code, you'll have to tweak.
IQueryable is a way to "slowly build up the query" instead of writing a single super query.......and helps with debugging. Eventually, you will comment out (or delete) ... tempDebuggingCollection ........ but it can help get you to where you want to go.
public async Task<ActionResult<object>> GetStarChemicalData(string starID)
{
IQueryable<YourObjectHere> starChemicalDataQueryable = await (from starlist in _context.StarList
join ql in _context.ChemicalList on starlist.ChemicalId equals ql.ChemicalId into stars
from chemicallist in stars.DefaultIfEmpty()
join qc in _context.ChemicalAtoms on chemicallist.ChemicalId equals qc.ChemicalId into chemicals
from chemicalatoms in chemicals.DefaultIfEmpty()
join nk in _context.StarLinks on chemicalatoms.AtomId equals nk.AtomId into links
from starlinks in links.DefaultIfEmpty()
where starlist.StarId == starID;
ICollection<YourObjectHere> tempDebuggingCollection = starChemicalDataQueryable.ToListAsync(CancellationToken.None);
var starChemicalData = starChemicalDataQueryable
select new
{
StarId = starlist.StarId,
StarType = starlist.StarType,
StarTitle = starlist.StarTitle,
ChemicalId = starlist.ChemicalId,
AtomId = (Guid?)chemicalatoms.AtomId,
OrderId = chemicalatoms.OrderId,
ChemicalText = chemicallist.ChemicalText,
AtomText = chemicalatoms.AtomText,
Wavelength = chemicalatoms.Wavelength,
isRedShifted = (starlinks.AtomId != null && starlist.StarType == 1) ? 1
: (starlinks.AtomId == null && starlist.StarType == 1) ? 0
: (int?)null
})
.GroupBy(x => x.StarId)
.Select(g => new
{
StarId = g.FirstOrDefault().StarId,
StarType = g.FirstOrDefault().StarType,
StarTitle = g.FirstOrDefault().StarTitle,
ChemicalId = g.FirstOrDefault().ChemicalId,
ChemicalText = g.FirstOrDefault().ChemicalText,
ChemicalAtoms = (g.FirstOrDefault().AtomId != null ? g.Select(x => new
{
AtomId = x.AtomId,
OrderId = x.OrderId,
AtomText = x.AtomText,
Feedback = x.Wavelength,
IsCorrect = x.isRedShifted
}) : null)
}).FirstOrDefaultAsync();
return starChemicalData;
............
AGain, same safe checks:
ChemicalAtoms = (g.FirstOrDefault().AtomId != null ? g.Select(x => new
{
AtomId = x.AtomId,
OrderId = x.OrderId,
AtomText = x.AtomText,
Feedback = x.Wavelength,
IsCorrect = x.isRedShifted
}) : null)
You are not safely checking for g.FirstOrDefault() ....
something like this:
ChemicalAtoms = null == g.FirstOrDefault() ? null : (g.FirstOrDefault().AtomId != null ? g.Select(x => new
{
AtomId = x.AtomId,
OrderId = x.OrderId,
AtomText = x.AtomText,
Feedback = x.Wavelength,
IsCorrect = x.isRedShifted
}) : null)

Mongodb processes only one record at a time?

I am using Mongodb in .net core.
At the same time I have many applications that query the same table. I have the problem that 1 record is processed in many applications.
I used the version in mongo but it didn't work well.
Here is my code:
try
{
TopupRequest topupReturn = null;
do
{
var topupRequest = await _paygateMongoRepository.GetOneAsync<TopupRequest>(p =>
p.Status == TopupStatus.Init && categoryCodes.Contains(p.CategoryCode) &&
p.ServiceCode == serviceCode);
if (topupRequest == null)
break;
var version = topupRequest.Version;
var versionPlus = topupRequest.Version + 1;
Expression<Func<TopupRequest, bool>> filter = p => p.Id == topupRequest.Id && p.Version == version;
var options = new FindOneAndUpdateOptions<TopupRequest, TopupRequest>
{
IsUpsert = false,
ReturnDocument = ReturnDocument.After
};
_logger.Info("Version: " + version + " version plus: " + versionPlus);
topupReturn = await _paygateMongoRepository.GetAndUpdateOne<TopupRequest, Guid>(filter,
Builders<TopupRequest>.Update.Set(m => m.Status, TopupStatus.InProcessing)
.Set(m => m.Version, versionPlus)
.Set(m => m.WorkerApp, workerApp), options);
_logger.Info("Version: " + version + " version plus: " + versionPlus);
} while (topupReturn == null);
if (topupReturn == null) return null;
if (timeout > 0 && topupReturn.CreatedTime.AddSeconds(timeout) <= DateTime.UtcNow) return null;
_logger.Info($"GetTopupPriorityAvailableVersionAsync success: {topupReturn.ToJson()}");
return topupReturn.ConvertTo<TopupRequestDto>();
}
catch (Exception e)
{
_logger.Info($"GetTopupPriorityAvailableVersionAsync error: {e}");
return null;
}
I used a transaction solution. But it also doesn't work for me.
using (var session = MongoDbContext.Client.StartSession())
{
var topups = MongoDbContext.GetCollection<TopupRequest>();
try
{
session.StartTransaction(new TransactionOptions(
readConcern: ReadConcern.Local,
readPreference: ReadPreference.Primary,
writeConcern: WriteConcern.WMajority));
var sort = Builders<TopupRequest>.Sort.Descending("CreatedTime");
var filter = Builders<TopupRequest>.Filter.Eq(p => p.Status, TopupStatus.Init)
& Builders<TopupRequest>.Filter.In("CategoryCode", categoryCodes)
& Builders<TopupRequest>.Filter.Eq(p => p.ServiceCode, serviceCode);
var options = new FindOneAndUpdateOptions<TopupRequest, TopupRequest>
{
IsUpsert = false,
ReturnDocument = ReturnDocument.After,
Sort = sort
};
var update = Builders<TopupRequest>.Update.Set("Status", TopupStatus.InProcessing);
var result = await topups.FindOneAndUpdateAsync(session, filter, update,options);
if (result == null)
session.AbortTransaction();
session.CommitTransaction();
return result;
}
catch (Exception e)
{
session.AbortTransaction();
Console.WriteLine(e);
return null;
}
}
I still had the same problem at the same time 1 record was used many times.
Please help me

Is it a lazy loading query or Eager Loading ? , Slow Query, Performance needed please

First, I would like to know if this query is Lazy loading or Eager loading. I Read a lot on both, and not sure if I understand the difference between each other.
2- I Get this query, This query take a lot of time to execute. Anybody have some suggest when you see this query. I'll do all modification needed to speed up this query.
Note: I just want your opinion about this query and method.
Thanks a lot.
public SearchLocationViewModel GetSearchLocationViewModel( string CivicNumber = null ,
string Street = null,
string City = null,
List<int?> ListCountryID = null,
List<int?> ListStateID = null,
int IsActive =1,
string SortField ="FileNumber",
string SortDirection = "asc" ,
List<int?> GrpDescID1 = null,
List<int?> GrpDescID2 = null,
List<int?> GrpDescID3 = null,
List<int?> GrpDescID4 = null,
int LocationTypeID = -1,
List<int?> ListUsageID = null)
{
if (GrpDescID1 == null)
{
GrpDescID1 = new List<int?>();
}
if (GrpDescID2 == null)
{
GrpDescID2 = new List<int?>();
}
if (GrpDescID3 == null)
{
GrpDescID3 = new List<int?>();
}
if (GrpDescID4 == null)
{
GrpDescID4 = new List<int?>();
}
if (ListCountryID == null)
{
ListCountryID = new List<int?>();
}
if (ListStateID == null)
{
ListStateID = new List<int?>();
}
if (ListUsageID == null)
{
ListUsageID = new List<int?>();
}
GrpDescID1.Remove(GrpDescID1.SingleOrDefault(p => p < 0));
GrpDescID2.Remove(GrpDescID2.SingleOrDefault(p => p < 0));
GrpDescID3.Remove(GrpDescID3.SingleOrDefault(p => p < 0));
GrpDescID4.Remove(GrpDescID4.SingleOrDefault(p => p < 0));
ListCountryID.Remove(ListCountryID.SingleOrDefault(p => p < 0));
ListStateID.Remove(ListStateID.SingleOrDefault(p => p < 0));
ListUsageID.Remove(ListUsageID.SingleOrDefault(p => p < 0));
int lang = BaseStaticClass.CurrentLangID();
int UserID = Convert.ToInt32(Session["UserID"]);
SearchLocationViewModel ViewModel = InitSearchViewModel();
IGrpRepository repGrp = new GrpRepository(_db);
ICountryRepository repCountry = new CountryRepository(_db);
IProvinceRepository repProvince = new ProvinceRepository(_db);
ViewModel.Perm = repPermission;
ViewModel. CivicNumber = CivicNumber ;
ViewModel. Street = Street;
ViewModel. City = City;
ViewModel. IsActive =IsActive;
ViewModel. SortField =SortField;
ViewModel. SortDirection = SortDirection ;
ViewModel.ListCountry = repCountry.GetCountryForSearchByUser(true,UserID);
ViewModel.ListProvince = repProvince.GetProvinceSearchByUserID(true, UserID);
ViewModel.ListGrpDescID1 =GrpDescID1;
ViewModel.ListGrpDescID2 = GrpDescID2;
ViewModel.ListGrpDescID3 = GrpDescID3;
ViewModel.ListGrpDescID4 = GrpDescID4;
ViewModel.ListCountryID = ListCountryID;
ViewModel.ListStateID = ListStateID;
ViewModel.LocationTypeID = LocationTypeID;
ViewModel.ListUsageID = ListUsageID;
var LocationType = new SelectList(repGeneric.GetTextByCurrentLang<LocationType, LocationTypeText>(), "ID", "Txt").ToList();
bc.AddDropdownSearchValueNoNew(ref LocationType);
var ListUsage = new SelectList(repGeneric.GetTextByCurrentLang<Usage, UsageText>(), "ID", "Txt").ToList();
ViewModel.ListUsage = ListUsage;
ViewModel.ListLocationType = LocationType;
var ListGrp1 = new SelectList(repGrp.GetAllGrpDescTextForUserByLevel(UserID, 1).AsEnumerable(), "GrpDescID", "GrpDescTxt").ToList();
var ListGrp2 = new SelectList(repGrp.GetAllGrpDescTextForUserByLevel(UserID, 2).AsEnumerable(), "GrpDescID", "GrpDescTxt").ToList();
var ListGrp3 = new SelectList(repGrp.GetAllGrpDescTextForUserByLevel(UserID, 3).AsEnumerable(), "GrpDescID", "GrpDescTxt").ToList();
var ListGrp4 = new SelectList(repGrp.GetAllGrpDescTextForUserByLevel(UserID, 4).AsEnumerable(), "GrpDescID", "GrpDescTxt").ToList();
var t1 = ListGrp1.Select(s => (int?)Convert.ToInt32(s.Value));
var t2 = ListGrp2.Select(s => (int?)Convert.ToInt32(s.Value));
var t3 = ListGrp3.Select(s => (int?)Convert.ToInt32(s.Value));
var t4 = ListGrp4.Select(s => (int?)Convert.ToInt32(s.Value));
ViewModel.ListGrp1 = ListGrp1;
ViewModel.ListGrp2 = ListGrp2;
ViewModel.ListGrp3 = ListGrp3;
ViewModel.ListGrp4 = ListGrp4;
ViewModel.ListGrpTogether = new List<SelectListItem>();
if(ViewModel.GrpName1 != "")
ViewModel.ListGrpTogether.Add(new SelectListItem() { Text = ViewModel.GrpName1 ,Value = "1"});
if (ViewModel.GrpName2 != "")
ViewModel.ListGrpTogether.Add(new SelectListItem() { Text = ViewModel.GrpName2, Value = "2" });
if (ViewModel.GrpName3 != "")
ViewModel.ListGrpTogether.Add(new SelectListItem() { Text = ViewModel.GrpName3, Value = "3" });
if (ViewModel.GrpName4 != "")
ViewModel.ListGrpTogether.Add(new SelectListItem() { Text = ViewModel.GrpName4, Value = "4" });
ViewModel.ListGrpTogether.Insert(0, new SelectListItem() { Text = ViewRes.GeneralString.Choose, Value = "-1", Selected = true });
int iUserID = Convert.ToInt32(Session["UserID"]);
//this is use for Permission
//Get all the user permission about group and province
IEnumerable<int?> usrGrpDesc = _db.UserGroupDescs.Where(p => p.UserID == iUserID).Select(s => s.GrpDescID);
IEnumerable<int?> usrProvince = _db.UserProvinces.Where(p => p.UserID == iUserID).Select(s => s.PrvID);
var ListLocation = from s in _db.Locations.Where(p =>
p.IsDelete == false &&
(IsActive < 0 || IsActive == (p.IsActive == true ? 1 : 0)) &&
(LocationTypeID < 0 || LocationTypeID == p.LocationTypeID) &&
(City == null || p.Address.City.CityName.Contains(City)) &&
(ListUsageID.Count() == 0 || p.Premises.Select(gs => gs.UsageID).Intersect(ListUsageID).Any()) &&
(ListCountryID.Count() == 0 || ListCountryID.Any(pl => pl == p.Address.City.Province.Country.CtryID)) &&
(ListStateID.Count() == 0 || ListStateID.Any(pl => pl == p.Address.City.Province.PrvID)) &&
(Street == null || p.Address.Street.Contains(Street)) &&
(CivicNumber == null || p.Address.CivicNumber.Contains(CivicNumber)) &&
((GrpDescID1.Count() == 0 )|| p.GroupLocations.Select(gs => gs.GrpDescID).Intersect(GrpDescID1).Any()) &&
((GrpDescID2.Count() == 0)|| p.GroupLocations.Select(gs => gs.GrpDescID).Intersect(GrpDescID2).Any()) &&
((GrpDescID3.Count() == 0) || p.GroupLocations.Select(gs => gs.GrpDescID).Intersect(GrpDescID3).Any()) &&
((GrpDescID4.Count() == 0 ) || p.GroupLocations.Select(gs => gs.GrpDescID).Intersect(GrpDescID4).Any()) &&
(p.GroupLocations.Select(gs => gs.GrpDescID).Intersect(usrGrpDesc).Any()) &&
((p.Address.City == null || usrProvince.Any(ps => ps.Value == p.Address.City.PrvID)))
)
select new LocationViewModel()
{
LocationID = s.LocationID,
LocationTypeID = s.LocationTypeID,
Long = s.Address.Longitude,
Lat = s.Address.Latitude,
FileNumber = s.LocationFile,
State = s.Address.City.Province.PrvName,
City = s.Address.City.CityName,
Address = s.Address.CivicNumber + " " + s.Address.Street,
Status = s.LocationType.LocationTypeTexts.Where(h => h.Txt != "" && h.LangID == lang || h.LangID == 1).OrderByDescending(g => g.LangID).FirstOrDefault().Txt,
ListGroupe1 = s.GroupLocations.Where(g=>g.GrpDesc.Grp.GrpLevel == 1).Select(grpLoc => grpLoc.GrpDesc.GrpDescTexts.Where(h => h.GrpDescTxt != "" && (h.LangID == lang || h.LangID == 1)).OrderByDescending(g => g.LangID).FirstOrDefault()).Select(txt => txt.GrpDescTxt),
ListGroupe2 = s.GroupLocations.Where(g => g.GrpDesc.Grp.GrpLevel == 2).Select(grpLoc => grpLoc.GrpDesc.GrpDescTexts.Where(h => h.GrpDescTxt != "" && (h.LangID == lang || h.LangID == 1)).OrderByDescending(g => g.LangID).FirstOrDefault()).Select(txt => txt.GrpDescTxt),
ListGroupe3 = s.GroupLocations.Where(g=>g.GrpDesc.Grp.GrpLevel == 3).Select(grpLoc => grpLoc.GrpDesc.GrpDescTexts.Where(h => h.GrpDescTxt != "" && (h.LangID == lang || h.LangID == 1)).OrderByDescending(g => g.LangID).FirstOrDefault()).Select(txt => txt.GrpDescTxt),
ListGroupe4 = s.GroupLocations.Where(g=>g.GrpDesc.Grp.GrpLevel == 4).Select(grpLoc => grpLoc.GrpDesc.GrpDescTexts.Where(h => h.GrpDescTxt != "" && (h.LangID == lang || h.LangID == 1)).OrderByDescending(g => g.LangID).FirstOrDefault()).Select(txt => txt.GrpDescTxt),
DefaultImgPath = s.LocationPictures.Where(p=>p.IsDefault == true && p.IsActive == true).FirstOrDefault().FilePath,
HasPremises = s.Premises.Any(p => p.IsActive == true && p.IsDelete == false)
};
ViewModel.ListLocation = ListLocation.ToList();
return ViewModel;
}
Lazy loading is deferring initialization of an object until the point at which it is needed. If you returned your ListLocation back to its caller, as you've written above, with no .ToList() (or other), then you'd be consuming this lazily.
Eager loading is having the results of your query gathered at the time that the query is defined. In this case it'd be retrieving the results of your LINQ query all at once (at the time that the query is constrcuted). Usually a .ToList() or .Single() or other will do this for you.
I suspect you're consuming the results your LINQ query (var ListLocation) later in your code. Your code above is using a lazy-loaded approach.
You're showing that you're calling .ToList(), so you're indeed using eager-loading; even though it's on a different statement/line of code.
Performance: I'm not 100% on this being your perf problem, but I'd refactor your LINQ into something like this, using an extension method .WhereIf(). It's a heck of a lot easier to read and write.
var ListLocation = from s in _db.Locations
.Where(p => p.IsDelete == false)
.WhereIf(IsActive >= 0, p=> IsActive == (p.IsActive == true ? 1 : 0))
.WhereIf(LocationTypeID >= 0, p=> LocationTypeID == p.LocationTypeID
.WhereIf(City!=null, p=> p.Address.City.CityName.Contains(City)) //etc
If you're using this, and it works, then you're probably lazy loading, since you don't have any calls to .Include().