I want to do a couple of db-operations in a single transaction.
It should be rolled back on all failures but one: in the middel of all the db-ops there will be an insert statement which inserts a generated number into a field that has an unique constraint.
If this insert fails because the generated number is already present, in should be repeated with a newly generated number until it succeeds. But this insert should not affect the other operations.
Since there are no nested tx in JPA, i am not sure what the best practice would be here. i am wondering if this is such an uncommon case.
I tried it with an "embedded" tx, which is opened with requires-new that suspends the surrounding tx and resumes it after successfully inserting the random number.
That worked partially: if there is an exception in the surrounding tx, everything is rolled back but the random number insert.
If i use one single tx, I don't know how i should determine why this tx failed to commit. so i don't know if i have to repeat all the operations with a new random number or leave it because of another failure.
I also tried to explicitely flush the entity manager session after the insert to force the exception before i continue with the rest of the work in hope i could just delete that entity from the EM programmatically and try again.
But I didn't find a way to remove that entity from the EM after a failed insert?
I am also using Spring and SpringDataJPA.
#Transactional
public void doSomeThing() {
SomeEntity fooEntity = new SomeEntity()
someEntityRepository.save(fooEntity);
someMoreDbOPs();
RandomNoEntity rndEnt = new RandomNoEntity();
while (true) {
try {
rndEnt.setRandomNumber(generateRandomNumber());
entityManager.persist(rndEnt);
entityManager.flush();
break;
} catch (ConstraintViolationException)
{
//entityManager.detach(rndEnt); // didnt work
//entityManager.remove(rndEnt); // didnt work
}
}
SomeEntity barEntity = new SomeEntity()
someEntityRepository.save(barEntity);
someMoreDbOPs();
}
i hope this pseudo code helps to describe what i am trying to do.
thanks in advance for any help...
Related
I was trying to manually implement a cascade delete in my ASP.NET Core website.
So, I got a Poll section which is based on 3 entities: PollQuestion, PollOption, PollAnswer.
I'm starting this by deleting all the answers, and this goes fine, then I move to deleting the options with this code:
if (answersDeleted) {
options = GetOptionsList(_pollID);
context.PollOption.RemoveRange(options);
context.SaveChanges();
return true;
} else {
return false;
}
When the SaveChanges() is executed I get this exception:
Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: Database operation expected to affect 1 row(s) but actually affected 2 row(s). Data may have been modified or deleted since entities were loaded.
So. I can't really understand why he was expecting to affect 1 row since I pass a list to RemoveRange().
In case someone could make use of this one day.... i found the problem, it wasn't strictly related to the chunk of code i posted; few days ago i modified the poll table, and changed the Key, from a single field key to a composite key, then i forgot to reflect that change in my Model, which still was using a single key. It wasn't returning any error so i didn't notice this until i had to do other stuff on the DB and there i found the problem.
I would like to be able to iterate through every row in an entity table without holding every row in memory. This is a read only operation and every row can be discarded after being processed.
If there is a way to discard the row after processing that would be fine. I know that this can be achieved using a DataReader (which is outside the scope of EF), but can it be achieved within EF?
Or is there a way to obtain a DataReader from within EF without directly using SQL?
More detailed example:
Using EF I can code:
foreach (Quote in context.Quotes)
sw.WriteLine(sw.QuoteId.ToString()+","+sw.Quotation);
but to achieve the same result with a DataReader I need to code:
// get the connection to the database
SqlConnection connection = context.Database.Connection as SqlConnection;
// open a new connection to the database
connection.Open();
// get a DataReader for our table
SqlCommand command = new SqlCommand(context.Quotes.ToString(), connection);
SqlDataReader dr = command.ExecuteReader();
// get a recipient for our database fields
object[] L = new object[dr.FieldCount];
while (dr.Read())
{
dr.GetValues(L);
sw.WriteLine(((int)L[0]).ToString() + "," + (string)L[1]);
}
The difference is that the former runs out of memory (because it is pulling in the entire table in the client memory) and the later runs to completion (and is much faster) because it only retains a single row in memory at any one time.
But equally importantly the latter example loses the Strong Typing of EF and should the database change, errors can be introduced.
Hence, my question: can we get a similar result with strongly typed rows coming back in EF?
Based on your last comment, I'm still confused. Take a look at both of below code.
EF
using (var ctx = new AppContext())
{
foreach (var order in ctx.Orders)
{
Console.WriteLine(order.Date);
}
}
Data Reader
var constr = ConfigurationManager.ConnectionStrings["AppContext"].ConnectionString;
using (var con = new SqlConnection(constr))
{
con.Open();
var cmd = new SqlCommand("select * from dbo.Orders", con);
var reader = cmd.ExecuteReader();
while (reader.Read())
{
Console.WriteLine(reader["Date"]);
}
}
Even though EF has few initial query, both of them execute similar query that can be seen from profiler..
I haven't tested it, but try foreach (Quote L in context.Quotes.AsNoTracking()) {...}. .AsNoTracking() should not put entities in cache so I assume they will be consumed by GC when they out of the scope.
Another option is to use context.Entry(quote).State = EntityState.Detached; in the foreach loop. Should have the similar effect as the option 1.
Third option (should definitely work, but require more coding) would be to implement batch processing (select top N entities, process, select next top N). In this case make sure that you dispose and create new context every iteration (so GC can eat it:)) and use proper OrderBy() in the query.
You need to use an EntityDataReader, which behaves in a way similar to a traditional ADO.NET DataReader.
The problem is that, to do so, you need to use ObjectContext instead of DbContext, which makes things harder.
See this SO answer, not the acepted one: How can I return a datareader when using Entity Framework 4?
Even if this referes to EF4, in EF6 things work in the same way. Usually an ORM is not intended for streaming data, that's why this functionality is so hidden.
You can also look at this project: Entity Framework (Linq to Entities) to IDataReader Adapter
I have done this by pages.
And cleaning the Context after each page load.
Sample:
Load first 50 rows
Iterate over them
Clean the Context or create a new one.
Load second 50 rows
...
Clean the Context = Set all its Entries as Detached.
I have a run into a serious problem several times now using MVC 4 and EF.
The problem is best illustrated by example:
I have records in a DB table with the following PKs, 1,2,3,4.
1 and 2 are deleted. When EF goes to insert a new record, it is assigning it with PK 1 again. The next insert will use 2 and then it will try 3 and get a PK violation.
I saw the same thing yesterday in another table with another insert.
In the following, image you can see that the db.SaveChanges failed when inserting the a new record with PK 3.
As you can see the DB is set to auto-increment from the following image:
Here is my controller action (it is used for inserts and edits - but that should not matter):
if (ModelState.IsValid)
{
//update pricelist
Pricelist pricelist = new Pricelist();
pricelist.InjectFrom(adminEditPricelistVM.Pricelist);
pricelist.PricelistProducts = new List<PricelistProduct>();
pricelist.SubscriberId = (int)UserManagement.getUsersSubscriberId(WebSecurity.GetUserId(System.Web.HttpContext.Current.User.Identity.Name));
if (adminEditPricelistVM.Pricelist.PricelistId != 0)
{
db.Entry(pricelist).State = EntityState.Modified;
}
else
{
db.Pricelists.Add(pricelist);
}
db.SaveChanges();
adminEditPricelistVM.Pricelist.PricelistId = pricelist.PricelistId;
etc...
The only clue I have is that in the seeding config for my data, we are using the following commands to begin the seeding at 1, when replacing the data:
context.Database.ExecuteSqlCommand("DBCC CHECKIDENT ('Features', RESEED, 1)");
Perhaps this has something to do with it, but I doubt it - since this is not called at that time.
BTW, this is not consistent. I cannot replicate manually. It just seems to happen from time-to-time and when it does, EF will continue to err on each insert attempt until it passes all the used IDs and finds the next free one. In other words, I will get an error inserting on PK 3. Then on the next insert attempt, it will err on PK 4 and then on the next attempt it will succeed because PK 5 was not being used. It's as if, there is a memory of PKs in use somewhere that gets reset.
Any help would be greatly appreciated!
OK. I can confirm that the problem is caused by the DB reseed. This basically sets the count back to 1 regardless of any remaining records in the DB. Weird but true.
I am using below code to insert the record into database,
EntityManagerFactory emf = getEmf();
em = emf.createEntityManager();
//Get the Transaction
EntityTransaction trx = em.getTransaction();
trx.begin();
for ( int i=0;i<10000;i++) {
//Create new Object and persist
Customer customer = new Customer(.....);
em.persist(customer);
//Once its reach the 1000 size do commit
if ( i % 1000 == 0 ) {
em.flush();
em.clear();
trx.commit();
}
}
em.close();
The above code is working fine when I don’t have any duplicate value in database as like new Customer object. If I have any duplicate value for Customer in database I am getting “SQLIntegrityConstraintViolationException”.
Here I am using persist () to insert the record into database for performance issue. If I use merge (), its working fine. but its need more time to complete the task.
In persist () case is there any option to identify which is duplicate record (i.e get duplicate record information)?
If more than one duplicate record present in DB, is there any option to identify which are duplicates records?
I believe avoiding adding duplicates is meant to be the application code's responsibility.
You could save a list of the customers you have sent to persist(), then prior to committing, query the db using a JPQL query like:
SELECT COUNT(c)
FROM Customer c
WHERE c in (:customers)
and send the list to the customers bind variable. Check ther result, and if the count is > 0, you know a customer has been duplicated and will cause a SQLIntegrityConstraintViolationException on commit.
Alternately, use a call to em.find(customer, Customer.class) prior to adding the em to the Persistence Context with the persist call. If it exists, don't add it.
I'm writing updates as part of CRUD testing and when I test my code, I get an error saying no entities are found. I have no idea why, because my partner did the exact same code and he worked perfectly. Neither of us is able to figure out what's going on. I'm getting an error on the getSingleResult() method.
#Test
public void updateBookTest() {
Book book = em.createQuery("select b from Book b where b.title = :title", Book.class).setParameter("title", "createABook").getSingleResult();
tx.begin();
book.setTitle("updatedThisBook");
book.setAuthor("newAuthor");
tx.commit();
Book updatedBook = em.find(Book.class, book.getBookId());
assertEquals(book.getTitle(), updatedBook.getTitle());
assertEquals(book.getAuthor(), updatedBook.getAuthor());
System.out.println("updateBookTest:\t" + book.toString());
tx.begin();
book.setTitle("createABook");
tx.commit();
}
This is my code. Let me know if more information is needed.
getSingleResult must to throw a NoResultException if there is no result.
So, your test is Ok.
Check that both are using same database (and there is no data returned), both are running same query, and both are using same jpa implementation versions.
From javadoc (since jpa 1.0):
getSingleResult
java.lang.Object getSingleResult(): Execute a SELECT query that returns a single untyped result.
Returns: the result
Throws:
NoResultException - if there is no result
NonUniqueResultException - if more than one result
IllegalStateException - if called for a Java Persistence query language UPDATE or DELETE statement
QueryTimeoutException - if the query execution exceeds the query timeout value set and only the statement is rolled back
TransactionRequiredException - if a lock mode has been set and there is no transaction
PessimisticLockException - if pessimistic locking fails and the transaction is rolled back
LockTimeoutException - if pessimistic locking fails and only the statement is rolled back
PersistenceException - if the query execution exceeds the query timeout value set and the transaction is rolled back
Reference javadoc getSingleResult
Another point, check that your friend is not calling getResultList instead of getSingleResult. This method returns a list and no throw an exception if empty.
Reference javadoc getResultList