I'm using Quarkus Reactive and I'm trying to insert data to DB if duplicate record is not found on DB, here's my line of code
#ApplicationScoped
public class MovieRepositories implements IMovieRepositories {
#Inject
MovieMapper mapper;
#Override
public Uni<Object> create(MovieEntity entity) {
final var data = mapper.toEntity(entity);
final var dataMovie = MovieEntity
.find("name=?1 and deleted_at is null", entity.getName());
return dataMovie.firstResult().onItem().ifNotNull()
.failWith(new ValidationException("Movie already exist"))
.call(c -> MovieEntity.persist(data))
.chain(d -> Uni.createFrom().item(data.getId()));
}
however, this code terminates after failure on this line of code
.failWith(new ValidationException("Movie already exist"))
and persist is never executed.
How to make this code insert data if no duplicate record is found on?
Thanks in advance
Solved by adding a return value to incoming message method that is calling this insert. This happens due to uncommited transaction to mongodb. By simply adding return commited the transaction to mongodb
Related
As per my understanding, committed records will never get roll back in ideal case. But committed records are really getting rolled back for the below code. I'm not sure this is done by JTA or by database which delete committed records when referenced parent table got roll back.
I will explain my scenario with the below simple code snippet. My transactional method createItems(List<items> items) will create list of parent items by calling itemService.saveParent(items). This will do flush() and then prepareChild(parentItems) will create thousands of child items for each parent items where child item will set primary key of parent item as its foreign key. Now I'm calling saveChild(subList) by making chunks (because I do not want to commit 1 million record with single transaction) and its transactional attribute is REQUIRES_NEW which will commit each sublist to DB. But one of the chunk got exception. When I checked the DB I saw that all inserted child records rolled back. How this happened for committed records ?
And If roll back would'n have happened, then how child records can exist when the parent records already got rolled back and no PK exist ?
#LocalBean
#Stateless
#TransactionAttribute(TransactionAttributeType.REQUIRED)
public class Job {
#EJB
private ItemService itemService;
public void createItems(List<items> items) {
List<Parent> parentItems = itemService.saveParent(items);
List<Child> childItems = itemService.prepareChild(parentItems);
for(List<Child> subList: getSublist(childItems )) {
itemService.saveChild(subList);
}
}
}
#LocalBean
#Stateless
public class ItemService {
#EJB
private ItemDLService itemDLService;
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void saveChild(List<child> childList) {
itemDLService.save(childList);
}
}
I have a requirement where i need to lookup few tables in ItemProcessor section. I dont want to make multiple JDBC call for each row in the ItemProcessor section where it might lead to performance issue when the spring batch started to process more number of records. What are the workarounds to avoid this situation? is there any way to preload these objects before the ItemProcessor or before batch starts and can refer it in ItemProcessor ?
You can use annotate your method with #PostConstruct to read data during the Spring application context initialization. Make your ItemReader's read method returns value from the list. When entire list is completed return null. This stops reading.
#Service
public class YourItemReader implements ItemReader<DomainObject> {
private int index;
List<DomainObject> dbRows;
#PostConstruct
public void init() {
List<DomainObject> //read from database
}
#Override
public DomainObject read(){
if (null != dbRows && index < dbRows.size()) {
return dbRows.get(index);
}
return null;
}
If the number of records are in millions, I would suggest to do a chunk based read from your database instead of reading all the records at once which might case Garbage collector out of memory exception. This can be done easily by adding a column called STATUS to your table to track the status of the records that are processed. Initially when you load data to your table, set the status as 'NOT PROCESSED' and when your ItemReader reads the chunk of records set the status to 'IN PROGRESS'. Once your ItemProcessor or ItemWriter completes its processing, change the status from 'IN PROGRESS' to 'PROCESSED'. Make sure to make the method which fetches the data from the database as 'synchronized'. This will make sure multiple threads not to fetch the same data from database.
public List<DomainObject> read(){
return fetchDataFromDb();
}
private synchronized List<DomainObject> fetchProductAssociationData(){
//read your chunk-size of records from database which has status as 'NOT
PROCESSED'
and change the status of the data which is read to 'IN PROGRESS'
return list;
}
When we pass our DbContext an object whose values have not changed, and try to perform an Update we get a 500 internal server error.
A user may open a dialog box to edit a record, change a value, change it back and then send the record to the database. Also we provide a Backup and Restore function and when the records are restored, some of them will not have changed since the backup was performed.
I was under the impression that a PUT would delete and re-create the record so I didn't feel there would be a problem.
For example, having checked that the Activity exists my ActivityController is as follows:
var activityEntityFromRepo = _activityRepository.GetActivity(id);
// Map(source object (Dto), destination object (Entity))
_mapper.Map(activityForUpdateDto, activityEntityFromRepo);
_activityRepository.UpdateActivity(activityEntityFromRepo);
// Save the updated Activity entity, added to the DbContext, to the SQL database.
if (await _activityRepository.SaveChangesAsync())
{
var activityFromRepo = _activityRepository.GetActivity(id);
if (activityFromRepo == null)
{
return NotFound("Updated Activity could not be found");
}
var activity = _mapper.Map<ActivityDto>(activityFromRepo);
return Ok(activity);
}
else
{
// The save failed.
var message = $"Could not update Activity {id} in the database.";
_logger.LogWarning(message);
throw new Exception(message);
};
My ActivityRepository is as follows:
public void UpdateActivity(Activity activity)
{
_context.Activities.Update(activity);
}
If any of the fields have changed then we don't get the error. Do I have to check every record for equality before the PUT? It seems unnecessary.
Perhaps I have missed something obvious. Any suggestions very welcome.
There is a lot of code missing here.
In your code you call your SaveChangesAsync (not the EF SaveChangesAsync).
Probably (but there is not the code to be sure) your SaveChangesAsync is something that returns false if there is an exception (and is not a good pattern because you "loose" the exception info) or if DbSet.SaveChangesAsync returns 0.
I think (but there is a lot of missing code) that this is your case. If you don't make any changes, SaveChangesAsync returns 0.
EDIT
The System.Exception is raised by your code (last line). EF never throws System.Exception.
I have a complex and big object graph that I want to insert in database by using a DbContext and SaveChanges method.
This object is a result of parsing a text file with 40k lines (around 3MB of data). Some collections inside this object have thousands of items.
I am able to parse the file correctly and add it to the context so that it can start tracking the object. But when I try to SaveChanges, it says:
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while updating the entries. See the inner exception for details. ---> System.Data.SqlClient.SqlException: String or binary data would be truncated.
I would like to know if there is a smart and efficient way of discovering which object is causing the issue. It seems that a varchar field is too little to store the data. But it's a lot of tables and fields to check manually.
I would like to get a more specific error somehow. I already configured an ILoggerProvider and added the EnableSensitiveDataLogging option in my dbContext to be able to see which sql queries are being generated. I even added MiniProfiler to be able to see the parameter values, because they are not present in the log generated by the dbContext.
Reading somewhere in the web, I found out that in EF6 there is some validation that happens before the sql is passed to the database to be executed. But it seems that in EF Core this is not available anymore. So how can I solve this?
After some research, the only approach I've found to solve this, is implementing some validation by overriding dbContext's SaveChanges method. I've made a merge of these two approaches to build mine:
Implementing Missing Features in Entity Framework Core - Part 3
Validation in EF Core
The result is...
ApplicationDbContext.cs
public override int SaveChanges(bool acceptAllChangesOnSuccess)
{
ValidateEntities();
return base.SaveChanges(acceptAllChangesOnSuccess);
}
public override async Task<int> SaveChangesAsync(bool acceptAllChangesOnSuccess, CancellationToken cancellationToken = new CancellationToken())
{
ValidateEntities();
return await base.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
}
private void ValidateEntities()
{
var serviceProvider = this.GetService<IServiceProvider>();
var items = new Dictionary<object, object>();
var entities = from entry in ChangeTracker.Entries()
where entry.State == EntityState.Added || entry.State == EntityState.Modified
select entry.Entity;
foreach (var entity in entities)
{
var context = new ValidationContext(entity, serviceProvider, items);
var results = new List<ValidationResult>();
if (Validator.TryValidateObject(entity, context, results, true)) continue;
foreach (var result in results)
{
if (result == ValidationResult.Success) continue;
var errorMessage = $"{entity.GetType().Name}: {result.ErrorMessage}";
throw new ValidationException(errorMessage);
}
}
}
Note that it's not necessary to override the other SaveChanges overloads, because they call these two.
The Error tells you that youre writing more characters to a field than it can hold.
This error for example would be thrown when you create a given field as NVARCHAR(4) or CHAR(4) and write 'hello' to it.
So you could simply check the length of the values you read in to find the one which is causing your problem. There is at least on which is too long for a field.
In Entity framework, I have implemented generic repository and unit of work pattern.
Below is related stuff from unit of work:
public IRepository<TEntity, TKey> GetRepository<TEntity, TKey>() where TEntity : class
{
if (_repositories == null)
{
_repositories = new Dictionary<string, object>();
}
string key = String.Format("{0}|{1}", typeof(TEntity).Name, typeof(TKey).Name);
if (_repositories.ContainsKey(key))
{
return (IRepository<TEntity, TKey>)_repositories[key];
}
Type repositoryType = typeof(Repository<TEntity, TKey>);
_repositories.Add(key, Activator.CreateInstance(repositoryType, _dataContext));
return (IRepository<TEntity, TKey>)_repositories[key];
}
From Manager layer, entity framework is call as below:
IRepository<tablenameEntity int> _tableEntityRepository = _unitOfWork.GetRepository<tablenameEntity, int>();
Error is as follow:
An error occurred while starting a transaction on the provider connection. See the inner exception for details.
{"New transaction is not allowed because there are other threads running in the session."}.
Actually, I just did as below,
remove stored procedure and function from model browser and added procedure and function again.
run custom tool on edmx file.
It just work out.
I am not sure, what was the issue. did that before also, but it was not work.. now, it works and not produce after that.
Thanks