I am getting an error below
Command aggregate failed: Sort exceeded memory limit of 104857600
bytes, but did not opt in to external sorting. Aborting operation.
Pass allowDiskUse:true to opt in..
Below is my sample code that has a lot of sorting and has big data. How do I allowDiskUse to true based on my sample below?
var collection = _context.GetCollection<SampleCollection>();
var result = collection.AsQueryable();
result = result.OrderByDescending(x => x.Date)
.ThenByDescending(d => d.Status)
.ThenByDescending(r => r.Firstname)
.ThenByDescending(d => d.Lastname)
.ThenByDescending(d => d.Birthdate)
.ThenByDescending(d => d.Sex);
return result .ToList();
I have updated my code to remove the AsQueryable.
var patientList = _context.GetCollection<CommunityGetPatients>();
var aggregateOptions = new AggregateOptions { AllowDiskUse = true };
var result = patientList.Aggregate(aggregateOptions).Match(x => x.Status != Status.Deleted)
.SortByDescending(x => x.Date)
.ThenByDescending(d => d.Status)
.ThenByDescending(r => r.Firstname)
.ThenByDescending(d => d.Lastname)
.ThenByDescending(d => d.Birthdate)
return result.ToList();
See below:
var client = new MongoClient();
var db = client.GetDatabase("db");
var collection = db.GetCollection<SampleCollection>("c");
var result = collection.AsQueryable<SampleCollection>(new AggregateOptions() { AllowDiskUse = true });
var res = result.OrderByDescending(x => x.Date)
.ThenByDescending(d => d.Status)
.ToList();
Related
I have the following code with annonymous type string date, I would like to cast it to List.
Thanks
var ReportDates = obj.Statuses.Where(x => x.StatusDate <= fourWeeksDate)
.GroupBy(x => x.StatusDate.ToString("dd-MMM"))
.Select((groupDate) => new { date = groupDate.Key }).ToList();
found the answer
var groupDate = "";
var ReportDates = obj.Statuses.Where(x => x.StatusDate <= fourWeeksDate)
.GroupBy(x => x.StatusDate.ToString("dd-MMM"))
.Select(x => groupDate = x.Key )
.ToList();
need to initialized outside
I want to moq update method that is using mongodbContext. here is what i am trying to do but its not working. how to pass UpdateResult return type .ReturnsAsync<UpdateResult>(). I am very new to Mongodb database Unit Testing with .net core.
public void UpdateEventAsync_Test()
{
//Arrange
var eventRepository = EventRepository();
var pEvent = new PlanEvent
{
ID = "testEvent",
WorkOrderID = "WorkOrderID",
IsDeleted = false,
IsActive = true,
EquipmentID = "EquipmentID"
};
////Act
mockEventContext.Setup(s => s.PlanEvent.UpdateOneAsync(It.IsAny<FilterDefinition<PlanEvent>>(), It.IsAny<UpdateDefinition<Model.EventDataModel.PlanEvent>>(), It.IsAny<UpdateOptions>(), It.IsAny<System.Threading.CancellationToken>())).ReturnsAsync<UpdateResult>();
var result = eventRepository.UpdateEventAsync(pEvent);
////Assert
result.Should().NotBeNull();
Assert.AreEqual(true, result);
}
below is the code for which i want to write Moq Test
public async Task<bool> UpdateEventAsync(Model.EventDataModel.PlanEvent eventobj)
{
var filter = Builders<Model.EventDataModel.PlanEvent>.Filter.Where(f => f.ID == eventobj.ID);
// TO Do: Use replace instead of update.
var updatestatement = Builders<Model.EventDataModel.PlanEvent>.Update.Set(s => s.IsDeleted, eventobj.IsDeleted)
.Set(s => s.EquipmentID, eventobj.EquipmentID)
.Set(s => s.PlannedStartDateTime, eventobj.PlannedStartDateTime)
.Set(s => s.PlannedEndDatetime, eventobj.PlannedEndDatetime)
.Set(s => s.WorkOrderID, eventobj.WorkOrderID)
.Set(s => s.ResourceID, eventobj.ResourceID)
.Set(s => s.LastUpdatedBy, eventobj.LastUpdatedBy)
.Set(s => s.EventComment, eventobj.EventComment)
.Set(s => s.SiteID, eventobj.SiteID)
.Set(s => s.LastUpdatedDateTime, DateTime.UtcNow.ToString());
UpdateResult updateResult = await _eventContext.PlanEvent.UpdateOneAsync(filter, updatestatement);
return updateResult != null && updateResult.IsAcknowledged && updateResult.ModifiedCount > 0;
}
Either create an instance or mock UpdateResult and return that from the setup
public async Task UpdateEventAsync_Test() {
//...omitted for brevity
var mockUpdateResult = new Mock<UpdateResult>();
//Set up the mocks behavior
mockUpdateResult.Setup(_ => _.IsAcknowledged).Returns(true);
mockUpdateResult.Setup(_ => _.ModifiedCount).Returns(1);
mockEventContext
.Setup(_ => _.PlanEvent.UpdateOneAsync(It.IsAny<FilterDefinition<PlanEvent>>(), It.IsAny<UpdateDefinition<Model.EventDataModel.PlanEvent>>(), It.IsAny<UpdateOptions>(), It.IsAny<System.Threading.CancellationToken>()))
.ReturnsAsync(mockUpdateResult.Object);
//Act
var result = await eventRepository.UpdateEventAsync(pEvent);
//Assert
result.Should().Be(true);
}
Also note that the test needs to be made async to be exercised accurately.
Before I asked Prevent sort result of union in entity framework
I got my answer but now I have new problem with this issue. I have this code:
var productExactlyTitle = products.Where(x => x.Title == keyword);
var productStartWithPhrase = products.Where(x => x.Title.StartsWith(keyword));
var productStartWithWord = products.Where(x => x.Title.StartsWith(keyword + " "));
var productContainsWord = products.Where(x => x.Title.Contains(" " + keyword + " "));
var productContainsPhrase = products.Where(x => x.Title.Contains(keyword)
|| x.Title.Contains(keyword)
|| x.SubTitle.Contains(keyword)
|| x.OtherName.Contains(keyword));
var splitWords = keyword.Split(' ');
var productSplitWordSearch = splitWords.Aggregate(products, (current, word) => current.Where(x => x.Title.Contains(word.Trim())));
var p1 = productExactlyTitle.Select(x => new { Item = x, Order = 1 });
var p2 = productStartWithWord.Select(x => new { Item = x, Order = 2 });
var p3 = productStartWithPhrase.Select(x => new { Item = x, Order = 3 });
var p4 = productContainsWord.Select(x => new { Item = x, Order = 4 });
var p5 = productContainsPhrase.Select(x => new { Item = x, Order = 5 });
var p6 = productSplitWordSearch.Select(x => new { Item = x, Order = 6 });
var productList = p1
.Union(p2)
.Union(p3)
.Union(p4)
.Union(p5)
.Union(p6)
.OrderBy(x => x.Order)
.Take(21)
.AsEnumerable()
.Select(x => new ProductItemViewModel()
{
Id = x.Item.Id,
Title = x.Item.Title,
Price = DiscountController.ApplyDiscountToPrice(x.Item).ToPrice(),
Image = x.Item.Images.FirstOrDefault(y => y.IsCoverPhoto)?.ImageUrl
});
Result of above code have duplicate records and I have to use select and distinct to remove duplicate records. so I change my code like this:
var productList = p1
.Union(p2)
.Union(p3)
.Union(p4)
.Union(p5)
.Union(p6)
.OrderBy(x => x.Order)
.Select(x => x.Item)
.Distinct()
.Take(21)
.AsEnumerable()
.Select(x => new ProductItemViewModel()
{
Id = x.Id,
Title = x.Title,
Price = DiscountController.ApplyDiscountToPrice(x).ToPrice(),
Image = x.Images.FirstOrDefault(y => y.IsCoverPhoto)?.ImageUrl
});
But after that my result is sorted with Id column again.
How can I solved this?
First, since adding Order field to each query makes the record unique, using Union (which is supposed to remove duplicates) doesn't make sense, so simply use Concat instead.
Second, to remove duplicates and not lose the Order field needed for later ordering, you need to group by Item and take the minimum Order for each group. The rest is the same as in the original solution.
var productList = p1
.Concat(p2)
.Concat(p3)
.Concat(p4)
.Concat(p5)
.Concat(p6)
.GroupBy(e => e.Item)
.Select(g => new { Item = g.Key, Order = g.Min(e => e.Order) })
.OrderBy(e => e.Order)
.Select(e => e.Item)
.Take(21)
.AsEnumerable()
...
In SQL server union, result is sorted based on primary key column. I want to prevent this behavior in entity framework.
In this post, #praveen has explained how to do this in pure sql. But I want to do this in entity framework.
My code:
public virtual ActionResult Search(string keyword)
{
var products = _db.Products
.Where(x => x.IsActive)
.AsQueryable();
var productExactlyTitle = products.Where(x => x.Title == keyword);
var productStartTitle = products.Where(x => x.Title.StartsWith(keyword));
var productContainsTitle = products.Where(x => x.Title.Contains(keyword)
|| x.Title.Contains(keyword)
|| x.SubTitle.Contains(keyword)
|| x.OtherName.Contains(keyword));
var productList = productExactlyTitle.Union(productStartTitle)
.Union(productContainsTitle)
.Take(10)
.AsEnumerable()
.Select(x => new ProductItemViewModel()
{
Id = x.Id,
Title = x.Title,
Price = x.Price.ToPrice(),
Image = x.Images.FirstOrDefault(y => y.IsCoverPhoto)?.ImageUrl
});
// some code ...
}
I want to show records with below order:
First: records of productExactlyTitle
Second: records of productStartTitle
Third: records of productContainsTitle
But result is sorted with Id column! and I don't want this.
Is there a way for do this?
In SQL all queries without an order by explicitly set is considered unordered. (and EF queries a translated into SQL). So if you want a specific order after your union just specify it.
var result = q1.Union(q2).OrderBy(x => x.?);
For your specific case:
var p1 = productExactlyTitle.Select(x => new { Item = x, Order = 1 });
var p2 = productStartTitle.Select(x => new { Item = x, Order = 2 });
var p3 = productContainsTitle.Select(x => new { Item = x, Order = 3 });
var productList = p1.Union(p2)
.Union(p3)
.OrderBy(x => x.Order)
.Select(x => x.Item)
.Take(10);
Task: use different where clause in one query
Here is example (it is not real query, just to illustrate the problem)
var events = ctx.Events; // ctx - EntityFramework context
var res = events
.GroupBy(ee => ee.State)
.Select(gg => new
{
State = gg.Key,
FirstTwo = events
// how to get this clause from variable
.Where(ee => ee.State == gg.Key)
.Take(2)
})
.ToList();
Next code did not work, the problem is that where expression use parameter from query gg.Key
var events = ctx.Events;
var res = events
.GroupBy(ee => ee.State)
.Select(gg => new
{
State = gg.Key,
FirstTwo = events
// 1
// how to get this clause from variable
//.Where(ee => ee.State == gg.Key)
// 2
// try to take out where expression from query
.Where(_buildExpression(gg.Key))
.Take(2)
})
.ToList();
// method
static Expression<Func<Event, bool>> _buildExpression(string state)
{
return ee => ee.State == state;
}
// exeption
An unhandled exception of type 'System.InvalidOperationException' occurred in EntityFramework.SqlServer.dll
Additional information: variable 'gg' of type 'System.Linq.IGrouping`2[System.String,Entities.Event]' referenced from scope '', but it is not defined
Example of getting where expression from variable, but does not depend on gg.Key (wrong)
Expression<Func<Event, bool>> whereClause = (ee) => (ee.State == "test");
var events = ctx.Events;
var res = events
.GroupBy(ee => ee.State)
.Select(gg => new
{
State = gg.Key,
FirstTwo = events
// 1
// how to get this clause from variable
//.Where(ee => ee.State == gg.Key)
// 2
// try to take out where expression from query
//.Where(_buildExpression(gg.Key))
// 3
// whereClause from variable, but does not depend on gg.Key
.Where(whereClause)
.Take(2)
})
.ToList();
How to take where сlause from variable with depend on gg.Key?
p.s. the query is just example of the problem. The code below does not solve the problem of real query:
var events = ctx.Events;
var res = events
.GroupBy(ee => ee.State)
.Select(gg => new
{
State = gg.Key,
FirstTwo = gg.Take(2)
})
.ToList();
Solution by OP.
Thanks to Ivan Stoev comment.
Expression<Func<Event, string, bool>> whereClause = (ee, state) => (ee.State == state);
var events = ctx.Events;
var res = events
.AsExpandable() // <= add this
.GroupBy(ee => ee.State)
.Select(gg => new
{
State = gg.Key,
FirstTwo = events
.Where(ee => whereClause.Invoke(ee, gg.Key)) // <= Invoke expression
.Take(2)
})
.ToList();
This was made possible by LinqKit