I am trying to create some simple sample data in my database. I have coded several IDataSeedContributorimplementations. In the implementations I inject and call multiple repositories to retrieve data seeded from my other implementations. I want to use that seeded data to populate the dependent properties of the entity in the current implementation.
Here is an example:
public ContactsDataSeedContributor(
IDepartmentRepository departmentRepository,
IContactRepository contactRepository,
IAddressRepository addressRepository,
IEmailAddressRepository emailAddressRepository,
IPhoneNumberRepository phoneNumberRepository,
IGuidGenerator guidGenerator,
ICurrentTenant currentTenant)
{
_departmentRepository = departmentRepository;
_contactRepository = contactRepository;
_addressRepository = addressRepository;
_emailAddressRepository = emailAddressRepository;
_phoneNumberRepository = phoneNumberRepository;
_guidGenerator = guidGenerator;
_currentTenant = currentTenant;
}
public async Task SeedAsync(DataSeedContext context)
{
using (_currentTenant.Change(context?.TenantId))
{
if (!(await _contactRepository.AnyAsync(x => x.TenantId.ToString() == _currentTenant.Id.ToString())))
{
var department = (await _departmentRepository.GetListAsync()).FirstOrDefault();
var contact = new Contact
(
id: _guidGenerator.Create(),
name: "",
surname: "Sample Contact",
title: "Sample Title",
lastContactDate: DateTime.UtcNow,
note: "This is a sample note.",
departmentId: department.Id,
clientId: null,
primary: true,
active: true
);
(await _addressRepository.GetListAsync()).ForEach(x => contact.Addresses.Add(x));
(await _emailAddressRepository.GetListAsync()).ForEach(x => contact.EmailAddresses.Add(x));
(await _phoneNumberRepository.GetListAsync()).ForEach(x => contact.PhoneNumbers.Add(x));
await _contactRepository.InsertAsync(contact);
}
}
}
I was anticipating and I have gotten several exceptions that demonstrate that the dependent data has not yet been initialized. That seems to suggest that there is not an intelligent mechanism for determining the order that the implementations should run.
Is there any way to give the framework a hint?
Can I run IDataSeedContributor implementations in a prescribed order?
I was trying to avoid a monolithic top-down implementation with everything running in my prescribed order.
Here is a class diagram for what I am trying to initialize:
I believe you can do this by injecting IDataSeeder instead of inheriting it from IDataSeedContributor directly. Because now you have complete control, you can call the SeedAsync method whenever and wherever you want.
I hope this information will be useful and you will achieve what you want in a short time.
References
https://docs.abp.io/en/abp/latest/Data-Seeding#idataseeder
While I appreciate that #berkansasmaz put me on the right path, it was not a complete solution. I thought I would share some of my solution here in hopes of helping someone else.
All of the examples given in the ABP documentation are for initializing entities that have no dependency on other entities. I wanted to create sample data so that when I create a new tenant, that tenant would already have something to work with.
My solution was to modify the provided SaasDataSeedContributor to include my own ITenantDataSeeder implementation.
public interface ITenantDataSeeder
{
Task CreateTenantAsync();
}
public class TenantDataSeeder : ITenantDataSeeder, ITransientDependency
{
private readonly ICurrentTenant _currentTenant;
private readonly ILogger<TenantDataSeeder> _logger;
private readonly ICountryDataSeeder _countryDataSeeder;
// other private fields here
public TenantDataSeeder(
ICountryDataSeeder countryDataSeeder,
IStateDataSeeder stateDataSeeder,
IAddressDataSeeder addressDataSeeder,
IEmailAddressDataSeeder emailAddressDataSeeder,
IPhoneNumberDataSeeder phoneNumberDataSeeder,
IDepartmentDataSeeder departmentDataSeeder,
IContactDataSeeder contactDataSeeder,
ICampaignDataSeeder campaignDataSeeder,
ILeadSourceDataSeeder leadSourceDataSeeder,
IProductGroupDataSeeder productGroupDataSeeder,
IProductDataSeeder productDataSeeder,
IClientNeedDataSeeder clientNeedDataSeeder,
IRevenueRepeatFreqDataSeeder repeatFreqDataSeeder,
IOpportunityStatusDataSeeder opportunityStatusDataSeeder,
IUserDataSeeder userDataSeeder,
IClientDataSeeder clientDataSeeder,
IOpportunityDataSeeder opportunityDataSeeder,
IOpportunityLineItemDataSeeder opportunityLineItemDataSeeder,
ISalesNoteDataSeeder salesNoteDataSeeder,
ICurrentTenant currentTenant,
ILogger<TenantDataSeeder> logger)
{
// private fields initialized here
}
public async Task CreateTenantAsync()
{
// Host Seeding
await _countryDataSeeder.CreateCountriesAsync();
if (!_currentTenant.IsAvailable)
return;
// Tenant Seeding
try
{
await _stateDataSeeder.CreateStatesAsync();
await _addressDataSeeder.CreateAddressesAsync();
await _emailAddressDataSeeder.CreateEmailAddressesAsync();
await _phoneNumberDataSeeder.CreatePhoneNumbersAsync();
await _departmentDataSeeder.CreateDepartmentsAsync();
await _contactDataSeeder.CreateContactsAsync();
await _campaignDataSeeder.CreateCampaignsAsync();
await _leadSourceDataSeeder.CreateLeadSourcesAsync();
await _productGroupDataSeeder.CreateProductGroupsAsync();
await _productDataSeeder.CreateProductsAsync();
await _clientNeedDataSeeder.CreateClientNeedsAsync();
await _repeatFreqDataSeeder.CreateRevenueRepeatFreqsAsync();
await _opportunityStatusDataSeeder.CreateOpportunityStatusesAsync();
//await _userDataSeeder.CreateUsersAsync();
await _clientDataSeeder.CreateClientsAsync();
await _opportunityDataSeeder.CreateOpportunitiesAsync();
await _opportunityLineItemDataSeeder.CreateOpportunityLineItemsAsync();
await _salesNoteDataSeeder.CreateSalesNotesAsync();
}
catch (ApplicationException e)
{
_logger.LogException(e, LogLevel.Warning);
}
}
}
And then I wrote several <Entity>DataSeeder interfaces and implementations.
NOTE that these classes DO NOT implement the IDataSeeder interface nor do they implement the IDataSeedContributor interface. I probably should re-factor them to be named <Entity>DataSeederFragmentor something equivalent to avoid confusion.
Here is an example of one of the interfaces and its implementation:
public interface IStateDataSeeder
{
Task<IEnumerable<State>> CreateStatesAsync();
}
public class StateDataSeeder : IStateDataSeeder, ITransientDependency
{
protected static IList<State> States = new List<State>();
private readonly ICurrentTenant _currentTenant;
private readonly IGuidGenerator _guidGenerator;
private readonly ICountryDataSeeder _countryDataSeeder;
private readonly IStateRepository _stateRepository;
public StateDataSeeder(
IStateRepository stateRepository,
ICountryDataSeeder countryDataSeeder,
IGuidGenerator guidGenerator,
ICurrentTenant currentTenant)
{
_stateRepository = stateRepository;
_countryDataSeeder = countryDataSeeder;
_guidGenerator = guidGenerator;
_currentTenant = currentTenant;
}
public async Task<IEnumerable<State>> CreateStatesAsync()
{
if (!_currentTenant.IsAvailable)
return Enumerable.Empty<State>();
try
{
var states = _stateRepository.Where(a => a.TenantId == _currentTenant.Id);
if (states.Any()) return states;
}
catch (NullReferenceException e)
{
throw new ApplicationException($"Separated tenant database not yet initialized. Run the DbMigrator.");
}
if (States.Any()) return States;
var usId = Guid.Parse(CountryConsts.IdUnitedStates);
var countries = await _countryDataSeeder.CreateCountriesAsync();
var usCountry = countries.FirstOrDefault(c => c.Id == usId);
if (usCountry == null)
throw new ApplicationException("Seeded Country United States was null.");
States.Add(await _stateRepository.InsertAsync(new State
(
id: _guidGenerator.Create(),
name: "Alabama",
code: "AL",
primary: false,
active: true,
countryId: usId
), autoSave: true));
// other states here
States.Add(await _stateRepository.InsertAsync(new State
(
_guidGenerator.Create(),
name: "Wyoming",
code: "WY",
primary: false,
active: true,
countryId: usId
), autoSave: true));
foreach (var state in States) state.TenantId = _currentTenant.Id;
return States;
}
}
NOTE: All of these interfaces and implementations were placed under my ProjectName.Domain project.
I am pretty sure that some of this could be done better. I debated about whether to use the static class field as a cache, but my downstream "data seeders" need to be able to access the states and the repository can't return them until they are committed by the [UnitOfWork].
I invite your constructive criticism. My purpose for posting this, however, was to help others like myself. I spent way too long trying to figure this out.
Related
I created a .net core web api project. It has gotten kinda big and I want to program a "delete" operation which deletes a lot of stuff from the database. Since there are a lot of things to delete, this will be a long running process. So I thought maybe I can run this in the background and just write status updates somewhere for the user to see whats happening.
I googled this and I found BackgroundWorkerQueue and thought this might be my solution.
So I registered the service and everything and here is my method that calls it:
public class DeleteController : ControllerBase {
private readonly BackgroundWorkerQueue _backgroundWorkerQueue;
public AdminController(BackgroundWorkerQueue backgroundWorkerQueue){
_backgroundWorkerQueue = backgroundWorkerQueue;
}
public async Task<ActionResult> HugeDeleteMethod(int id)
{
// some prechecks here...
// and here I thought I'd start the background task
_backgroundWorkerQueue.QueueBackgroundWorkItem(async token =>
{
var a = _context.StatusTable.Find(id);
a.Status += "Blablablabla\n";
_context.StatusTable.Update(a);
await _context.SaveChangesAsync();
//now start doing delete operations
});
}
}
And that class looks like this:
public class BackgroundWorkerQueue
{
private ConcurrentQueue<Func<CancellationToken, Task>> _workItems = new ConcurrentQueue<Func<CancellationToken, Task>>();
private SemaphoreSlim _signal = new SemaphoreSlim(0);
public async Task<Func<CancellationToken, Task>> DequeueAsync(CancellationToken cancellationToken)
{
await _signal.WaitAsync(cancellationToken);
_workItems.TryDequeue(out var workItem);
return workItem;
}
public void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem)
{
if (workItem == null)
{
throw new ArgumentNullException(nameof(workItem));
}
_workItems.Enqueue(workItem);
_signal.Release();
}
}
There is also a DeleteService, which is also called in my startup, but I am not sure what it does:
public class DeleteService : BackgroundService
{
private readonly BackgroundWorkerQueue queue;
public NukeService(BackgroundWorkerQueue queue)
{
this.queue = queue;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
var workItem = await queue.DequeueAsync(stoppingToken);
await workItem(stoppingToken);
}
}
}
Both are added in my startup.cs:
services.AddHostedService<DeleteService>();
services.AddSingleton<BackgroundWorkerQueue>();
Well, maybe I'm going about this all wrong. This is never called it seems, the StatusTable field "Status" is always empty. So how do I do this?
You just need to subclass BackgroundService class or implement IHostedService and than register your service as hosted service.
This will run a service in the background. Than in your service you can leverage the BlockingQueue that will perform tasks only when they are added, e.g. like this:
public class MyService : BackgroundService {
private readonly BlockingCollection<long> queue;
public MyService(){
this.queue = new BlockingCollection<long>();
Task.Run(async () => await this.Execute());
}
public void AddId(long id) {
this.queue.Add(id);
}
private async Task Execute()
{
foreach (var id in this.queue.GetConsumingEnumerable())
{
... do your stuff ...
}
}
}
services.AddHostedService<MyService>();
Here is the docu: Background services in .net core
I have a little problem. But I dont know why it doesnt work. And I dont know how to post all ids by postman.
I am using unit of work with generic repository. I want to send int[] ids to my controller. I dont want to send entity. I searched a lot it today. And I changed my code. But what is problem now?
This is my repostiroy:
public async Task DeleteRangeAsync(Expression<Func<T, bool>> predicate)
{
IQueryable<T> query = _dbSet.Where(predicate);
await Task.Run(() => { _dbSet.RemoveRange(query.AsNoTracking()); });
}
This is my KulturManager:
public async Task<IResult> HardDeleteRangeAsync(int[] ids)
{
await UnitOfWork.Kulturs.DeleteRangeAsync(c => ids.Contains(c.Id));
await UnitOfWork.SaveAsync();
return new Result(ResultStatus.Success, Messages.Info("Kultur", "HardDelete"));
}
And this is my KulturController:
[HttpDelete("{ids}")]
public async Task<IActionResult> HardDeleteRangeAsync(int[] ids)
{
var result = await _kulturManager.HardDeleteRangeAsync(ids);
return Ok(result.Message);
}
Thank you for help
You shouldn't fetch all the entities you want to delete. Instead create stub entities for RemoveRange. If you don't have a common base class, this requires reflection, but with a common entity base class you can do it like this:
public void DeleteRange<T>(int[] ids) where T: BaseEntity, new()
{
_dbSet.RemoveRange(ids.Select(i => new T() { Id = i }).ToList());
}
or if the method is in a generic class, the method would look like
public void DeleteRange(int[] ids)
{
_dbSet.RemoveRange(ids.Select(i => new T() { Id = i }).ToList());
}
And there's no reason to mark this as Async now since it doesn't do any database access.
We are using ASP.NET Zero and are running into issues with parallel processing from an AppService. We know requests must be transactional, but unfortunately we need to break out to slow running APIs for numerous calls, so we have to do parallel processing.
As expected, we are running into a DbContext contingency issue on the second database call we make:
System.InvalidOperationException: A second operation started on this context
before a previous operation completed. This is usually caused by different
threads using the same instance of DbContext, however instance members are
not guaranteed to be thread safe. This could also be caused by a nested query
being evaluated on the client, if this is the case rewrite the query avoiding
nested invocations.
We read that a new UOW is required, so we tried using both the method attribute and the explicit UowManager, but neither of the two worked.
We also tried creating instances of the referenced AppServices using the IocResolver, but we are still not able to get a unique DbContext per thread (please see below).
public List<InvoiceDto> CreateInvoices(List<InvoiceTemplateLineItemDto> templateLineItems)
{
List<InvoiceDto> invoices = new InvoiceDto[templateLineItems.Count].ToList();
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
Parallel.ForEach(templateLineItems, async (templateLineItem) =>
{
try
{
XAppService xAppService = _iocResolver.Resolve<XAppService>();
InvoiceDto invoice = await xAppService
.CreateInvoiceInvoiceItem();
invoices.Insert(templateLineItems.IndexOf(templateLineItem), invoice);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});
if (exceptions.Count > 0) throw new AggregateException(exceptions);
return invoices;
}
How can we ensure that a new DbContext is availble per thread?
I was able to replicate and resolve the problem with a generic version of ABP. I'm still experiencing the problem in my original solution, which is far more complex. I'll have to do some more digging to determine why it is failing there.
For others that come across this problem, which is exactly the same issue as reference here, you can simply disable the UnitOfWork through an attribute as illustrated in the code below.
public class InvoiceAppService : ApplicationService
{
private readonly InvoiceItemAppService _invoiceItemAppService;
public InvoiceAppService(InvoiceItemAppService invoiceItemAppService)
{
_invoiceItemAppService = invoiceItemAppService;
}
// Just add this attribute
[UnitOfWork(IsDisabled = true)]
public InvoiceDto GetInvoice(List<int> invoiceItemIds)
{
_invoiceItemAppService.Initialize();
ConcurrentQueue<InvoiceItemDto> invoiceItems =
new ConcurrentQueue<InvoiceItemDto>();
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
Parallel.ForEach(invoiceItemIds, (invoiceItemId) =>
{
try
{
InvoiceItemDto invoiceItemDto =
_invoiceItemAppService.CreateAsync(invoiceItemId).Result;
invoiceItems.Enqueue(invoiceItemDto);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});
if (exceptions.Count > 0) {
AggregateException ex = new AggregateException(exceptions);
Logger.Error("Unable to get invoice", ex);
throw ex;
}
return new InvoiceDto {
Date = DateTime.Now,
InvoiceItems = invoiceItems.ToArray()
};
}
}
public class InvoiceItemAppService : ApplicationService
{
private readonly IRepository<InvoiceItem> _invoiceItemRepository;
private readonly IRepository<Token> _tokenRepository;
private readonly IRepository<Credential> _credentialRepository;
private Token _token;
private Credential _credential;
public InvoiceItemAppService(IRepository<InvoiceItem> invoiceItemRepository,
IRepository<Token> tokenRepository,
IRepository<Credential> credentialRepository)
{
_invoiceItemRepository = invoiceItemRepository;
_tokenRepository = tokenRepository;
_credentialRepository = credentialRepository;
}
public void Initialize()
{
_token = _tokenRepository.FirstOrDefault(x => x.Id == 1);
_credential = _credentialRepository.FirstOrDefault(x => x.Id == 1);
}
// Create an invoice item using info from an external API and some db records
public async Task<InvoiceItemDto> CreateAsync(int id)
{
// Get db record
InvoiceItem invoiceItem = await _invoiceItemRepository.GetAsync(id);
// Get price
decimal price = await GetPriceAsync(invoiceItem.Description);
return new InvoiceItemDto {
Id = id,
Description = invoiceItem.Description,
Amount = price
};
}
private async Task<decimal> GetPriceAsync(string description)
{
// Simulate a slow API call to get price using description
// We use the token and credentials here in the real deal
await Task.Delay(5000);
return 100.00M;
}
}
I am having difficulty understanding the documentation for the Audit.NET Entity Framework Data Provider, to save Audit.NET WebAPI audit logs to my database.
This is how I have my Audit configuration set, just to test. I have a breakpoint inside the AuditEntityAction on entity.ChangeType = ev.EventType, but this never gets hit when I call an audited action on my controller.
Audit.Core.Configuration.Setup()
.UseEntityFramework(x =>
x.AuditTypeMapper(t => typeof(AuditLog))
.AuditEntityAction<AuditLog>((ev, entry, entity) =>
{
entity.ChangeType = ev.EventType;
entity.ObjectType = entry.EntityType.Name;
entity.PrimaryKey = "test";
entity.TableName = "test";
entity.UserId = entry.CustomFields[UserIdField].ToString();
})
.IgnoreMatchedProperties()
);
On my controller action, I have the decorator:
[AuditApi(EventTypeName = "Organisation:Create", IncludeRequestBody = true, IncludeResponseBody = true)]
Is this correct? I am not very clear on this, and I would appreciate some pointers.
The Entity Framework Data Provider is part of the library Audit.EntityFramework and was designed to exclusively store the audits that are generated by an audited Entity Framework DbContext.
So it will not work for WebApi events of any other kind of event.
Here you can see how the audit event is discarded if it's not an AuditEventEntityFramework
So you should create your own Custom Data Provider or maybe use the SQL Data Provider.
You can use Audit.NetWebApi package to get the WebApiAudit logs
public static void UseAudit(this IApplicationBuilder app, IHttpContextAccessor contextAccessor)
{
Audit.Core.Configuration.AddCustomAction(ActionType.OnScopeCreated, scope =>
{
var entityTrack = scope.Event.GetEntityFrameworkEvent();
var requestTrack = scope.Event.GetWebApiAuditAction();
if (entityTrack!=null)
{
foreach (var item in entityTrack.Entries)
{
scope.Event.CustomFields[Table] = item.Table;
scope.Event.CustomFields[Action] = item.Action;
}
}
else if(requestTrack!=null)
{
scope.Event.CustomFields[Action] = $"{requestTrack.ActionName}:{requestTrack.ActionName}";
scope.Event.CustomFields[RequestBody] = requestTrack.RequestBody.Value.ToString();
scope.Event.CustomFields[ResponseBody] = requestTrack.ResponseBody?.Value?.ToString()?? string.Empty;
scope.Event.CustomFields[Exception] = requestTrack.Exception?? string.Empty;
}
});
}
And then put this function in Startup.cs ConfigureApp
public void Configure(IApplicationBuilder app, IHostingEnvironment env, IHttpContextAccessor contextAccessor)
{
app.UseAudit(contextAccessor);
}
Constants used:
private const string Table = "Table";
private const string Action = "Action";
private const string RequestBody = "RequestBody";
private const string ResponseBody = "ResponseBody";
private const string Exception = "Exception";
I have a question that is similar to this one.
However none of the answers are going to be acceptable in my situation. I have implemented several of the classes that Microsoft suggests here and have been struggling with a scenario that I feel is quite common.
I need to test that an item that is being created in a controller's action is correctly being saved to the database. Here's the controller method:
[HttpPost]
[ActionName("CreateUser")]
public virtual async Task<IHttpActionResult> CreateUser(RegisterViewModel viewModel)
{
ApplicationUser user = new ApplicationUser(false, true, true, viewModel.UserName,
viewModel.FirstName, viewModel.LastName, viewModel.EmailAddress);
try
{
user.Salt = HashUtilities.GetSalt();
user.Password = HashUtilities.GetHashedValue(viewModel.Password, user.Salt);
// This is what I want to assert happened
_dbContext.ApplicationUsers.Add(user);
await _dbContext.SaveChangesAsync();
return Ok(user);
}
catch (DbEntityValidationException ex)
{
List<string> messages = new List<string>();
foreach (DbEntityValidationResult eve in ex.EntityValidationErrors)
foreach (DbValidationError dve in eve.ValidationErrors)
messages.Add(dve.ErrorMessage);
string message = string.Join("\n\n", messages);
return BadRequest(message);
}
catch (Exception ex)
{
return InternalServerError(ex);
}
}
And here is my test. Some of the class/method implementations such as InMemoryAsyncQueryable are directly copied from MS's suggestion on the codeplex link.
// Arrange
var viewModel = new RegisterViewModel
{
ConfirmPassword = "abc123",
EmailAddress = "Test#test.com",
FirstName = "Test",
LastName = "User1",
Password = "abc123",
UserName = "TestUser1"
};
var queryableUsers = new List<ApplicationUser>().AsQueryable();
var users = DataUtilities.CreateMockQueryableSet<ApplicationUser>(new InMemoryAsyncQueryable<ApplicationUser>(queryableUsers));
var mockContext = new Mock<MainContext>();
mockContext.Setup(x => x.ApplicationUsers).Returns(users.Object);
var controller = new ApplicationUserController(mockContext.Object);
// Act
var result = controller.CreateUser(viewModel).Result;
// Assert
Assert.IsInstanceOf<OkNegotiatedContentResult<ApplicationUser>>(result);
mockContext.Verify(x => x.SaveChangesAsync());
// Test for persistence to the DbSet this is what is breaking down.
// My ApplicationUsers.Count() is 0 when I'm expecting 1.
Assert.AreEqual(mockContext.Object.ApplicationUsers.Count(), 1);
var testUser = mockContext.Object.ApplicationUsers.First();
Frankly, I've found that testing this kind of behavior is really difficult. I'd advise you to take a look at a solution for DbContext wrapping in memory, such as the Effort library.
I explained how I use it in this answer: it was for Entity Framework but I think the same behavior can be used in your case.