I'm designing JAX-RS APIs.
POST /myentities
PUT /myentities/{myentityId}
I did like this.
#PUT
#Path("/{myentityId: \\d+}")
#Consumes(...)
#Transactional
public Response updateMyentity(
#PathParam("myentityId") final long myentityId,
final Myentity myentity) {
if (!Objects.equal(myentitiyId, myentity.getId())) {
// throw bad request
}
entityManager.merge(myentity);
return Response.noContent().build();
}
I suddenly get curious and questioned to my self.
When a client invokes following request
PUT /myentities/101 HTTP/101
Host: ..
<myentity id=101>
</myentity>
Is it possible that the request is processed even if there is no resource identified by 101?
I ran a test.
acceptEntityManager(entityManager -> {
// persist
final Device device1 = mergeDevice(entityManager, null);
entityManager.flush();
logger.debug("device1: {}", device1);
assertNotNull(device1.getId());
// remove
entityManager.remove(device1);
entityManager.flush();
assertNull(entityManager.find(Device.class, device1.getId()));
// merge again, with non existing id
final Device device2 = entityManager.merge(device1);
entityManager.flush();
logger.debug("device2: {}", device2);
});
I found that second merge works and a new id assigned.
Is this normal? Shouldn't EntityManager#merge deny the operation?
Is it mean that any client can attack the API by calling
PUT /myentities/<any number>
?
If I understand this right, then yes, this is how merge should work.
merge creates a new instance of your entity class, copies the current state of the entity that you provide (in your case device1, and adds that NEW instance to the transaction.
So the NEW instance is managed, and any changes you make to the old instance are not updated in the transaction and will not be propagated to the database via flush - operation.
So while you removed device1 from the database with entityManager.remove(device1); the call
final Device device2 = entityManager.merge(device1); adds a new copy of it (with new ID) to the database. Changes you make to device1 are not tracked in the transaction, so they won't be reflected in the database tables once you flush. Changes to device2 on the other hand WILL be tracked in the transaction.
Related
I have this API which deletes and adds different users into the the Database. The way this API is written is something like this:
/* Assuming 'users' is an array of users to delete (if users[x].toDelete == true),
or to add otherwise */
using (var db = new myContext())
{
using (var dbTransaction = db.Database.BeginTransaction())
{
try
{
/* Performing deletion of Users in DB */
SaveChanges();
/* Performing add of Users in DB */
foreach(user in users)
if (!user.toDelete) {
db.users.add(..);
db.SaveChanges();
}
dbTransaction.Commit();
}
catch (ValidationException ex)
{
dbTransaction.Rollback();
}
}
}
The reason why I use a transaction is that I have some validations that should be run on the new data.. For example, I cannot add two user with the same email!
All the validation Server-Side is done with Data-Annotations and the MetaData classes, therefore I created the Attribute Uniqueness which I associated to the property email of the class UserMetaData.
So, the problem is that, inside the Uniqueness Attribute I need to check again the database to search for other users with the same email:
public class IsUnique : ValidationAttribute
{
protected override ValidationResult IsValid(object value, ValidationContext validationContext)
{
...
using (var db = new myContext()) {
/* Here I perform a select on the DB looking for records
with the same email, all using Reflection. */
if (recordFound > 0)
return new ValidationResult("email already exists");
return ValidationResult.Success;
}
}
}
So, as you can see, inside the validation I use new myContext() and that's where the problem is: let's pretend I have an empty database and I add two user with the same email in the same query.
As you can see, I call the db.SaveChanges() after every user has been added, therefore I thought that, when the second user to be added would have been validated, my validation would have known that the first user added and the second one have the same email, therefore an Validation Exception would have been throw and the dbTransaction.Rollback() called.
The problem is that inside the validator i call new myContext() and inside that context the changes that I thought would have been there thanks to the db.SaveChanges() there arent't, and that's I think is because all the changes are inside a Transaction.
Sooo.. The solutions I thought so far are two:
Nested Transaction: in the API I have an outer transaction which save the state of the DB at the beginning, then some inner transaction are run every time a db.SaveChanges() is called. Doing this, the validation does see the changes, but the problem is that, if in the catch i called outerTransaction.rollback(), the changes made by the inner transactions are not rollbacked;
Some way to retrieve the context I have used in the API even in the validation.. But in there I only have these two arguments, so I don't know if it's even possible;
When we pass our DbContext an object whose values have not changed, and try to perform an Update we get a 500 internal server error.
A user may open a dialog box to edit a record, change a value, change it back and then send the record to the database. Also we provide a Backup and Restore function and when the records are restored, some of them will not have changed since the backup was performed.
I was under the impression that a PUT would delete and re-create the record so I didn't feel there would be a problem.
For example, having checked that the Activity exists my ActivityController is as follows:
var activityEntityFromRepo = _activityRepository.GetActivity(id);
// Map(source object (Dto), destination object (Entity))
_mapper.Map(activityForUpdateDto, activityEntityFromRepo);
_activityRepository.UpdateActivity(activityEntityFromRepo);
// Save the updated Activity entity, added to the DbContext, to the SQL database.
if (await _activityRepository.SaveChangesAsync())
{
var activityFromRepo = _activityRepository.GetActivity(id);
if (activityFromRepo == null)
{
return NotFound("Updated Activity could not be found");
}
var activity = _mapper.Map<ActivityDto>(activityFromRepo);
return Ok(activity);
}
else
{
// The save failed.
var message = $"Could not update Activity {id} in the database.";
_logger.LogWarning(message);
throw new Exception(message);
};
My ActivityRepository is as follows:
public void UpdateActivity(Activity activity)
{
_context.Activities.Update(activity);
}
If any of the fields have changed then we don't get the error. Do I have to check every record for equality before the PUT? It seems unnecessary.
Perhaps I have missed something obvious. Any suggestions very welcome.
There is a lot of code missing here.
In your code you call your SaveChangesAsync (not the EF SaveChangesAsync).
Probably (but there is not the code to be sure) your SaveChangesAsync is something that returns false if there is an exception (and is not a good pattern because you "loose" the exception info) or if DbSet.SaveChangesAsync returns 0.
I think (but there is a lot of missing code) that this is your case. If you don't make any changes, SaveChangesAsync returns 0.
EDIT
The System.Exception is raised by your code (last line). EF never throws System.Exception.
I have read this post and the theory I think that is clear. I have a DAL that only has the methods to add, get, update and delete information in a database.
So I guess that I have an application in which I have clients, orders and type of client. Type of client has a percent that set the discount to make to a type of client.
The business layer request to DAL the type of client to know the discount.
The business layer create the order with the price and apply the discount according to the type of client.
The business layer send to the DAL the command to add the new order, sending the new order.
In code I can have this:
DAL:
public async getClientType(long paramIDClientType)
{
using(Entities myDbContext = new Entities())
{
return await myDbContext.ClientTypes.Where(x=> x.IDType == paramIDClientType).SingleOrDefault();
}
}
public async addOrder(Orders paramNewOrder)
{
using(Entities myDbContext = new Entities())
{
myDbContext.Orders.Local.Add(paramNewOrder);
await myDbContext.SaveChangesAsync();
}
}
Business layer:
public void addOrderToClient(CLients paramClient)
{
ClientTypes myType = myDAL.getClientType(paramClient.IDClient);
ORder myNewOrder = myNewOder();
myNewOrder.IDClient = paramClientIdCLient;
myNewOder.Amount = 300;
myNewOrder.Discount = myType.Discount;
myNewOder.Total = nyNewOrder.Total - myNewOder.Amount * myNewOder.Discount;
myDAL.AddOrder(nyNewOrder);
}
But I have a problem with the concurrency in this case, because I want to ensure that I use the correct discount, so I want to avoid the discount of a type of client is changed by another user un the middle of the process of add the new order.
If I use optimistic concurrency, I have to have a timestamp column in my ClientTypes table, but this not solve my problem, because in the addOrder method in my DAL layer, I only pass as parameter the new order, so the method don't have the timestamp value that has the business layer to check if the type of the client has changed to ensure that the discount used is the correct.
SO I am thinking in this solution:
public async addOrder(Orders paramNewOrder)
{
using(Entities myDbContext = new Entities())
{
string sql = "select ct.* from ClientTypes as ct, CLients as c"
+ " where ct.IdType = c.IdType and c.IdType = " + paramNewOrder.IdCLient;
ClientTypes myClientType = await myDbContext.CLientTypeSqlQuery<CLientTypes>(sql).SingleOrDefaultAsync();
if(paramNewOrder.Discount != myCLientType)
{
throw new Exception("Discount incorrect.");
}
paramNewOrder.Total = paramNewOrder.Amount - paramNewOrder.Amount * myClientType.Discount;
myDbContext.Orders.Local.Add(paramNewOrder);
await myDbContext.SaveChangesAsync();
}
}
This is my business layer, but use EF to get the data, so I think that this solution merge DAL abd business layer. Is this true? If this is true, I guess that is a not good solution. But then, how could I control concurrency?
Thanks.
Yes, optimistic concurrency control doesn't help you to prevent inserting a faulty new order, because you don't commit the ClientType. Only updating a ClientType would raise an exception if the discount was changed in the mean time.
But carefully consider the requirements. Is it really of paramount importance that the correct discount is used milliseconds after it's modified? If so, you have to look for a locking mechanism. Otherwise, just fetch the current discount at the very last moment, do the calculation and commit the order.
You could implement a locking/calculation/insert mechanism in a stored procedure that is mapped to the insert action of an Order. EF can map CUD actions to stored procedures..
I am writing a web application, such that I get different objects back from the web that need to be either updated or added to the database. On top of this, I need to check that the owner is not modified. Since a hacker could potentially get an account and send an update to modify the foreign key to the user model. I don't want to have to manually code all of these methods, instead I want to make a simple generic call.
Maybe something as simple as this
ctx.OrderLines.AddOrUpdateSet(order.OrderLines, a => a.Order)
Based on old persisted records that have a foreign key to Order, and on the new incoming records.
Delete old records that are not on the new records list.
Add new records that are not on the old records list.
Update new records that exist on both lists.
ctx.Entry(orderLine).State=EntityState.Deleted;
...
ctx.Entry(orderLine).State=EntityState.Added;
...
ctx.Entry(orderLine).State=EntityState.Modified;
This gets a bit complicated when the old record is loaded to verify that ownership did not change. I get an error if I don't do.
oldorder.OrderLines.remove(oldOrderLine); //for deletes
oldorder.OrderLines.add(oldOrderLine); //for adds
ctx.Entry(header).CurrentValues.SetValues(header); //for modifications
With Entity Framework 5 there is a new extension function called AddOrUpdate. And there was a very interesting (please read) blog entry on how to create this method before it was added.
I'm not sure if this is too much to ask as a question in StackOverflow, any clues on how to approach the problem may be sufficient. Here are my thoughts so far:
a) leverage AddOrUpdate for some of the functionality.
b) create a secondary context hoping to avoid loading order into the context and avoid extra calls.
c) Set the state of all the saved objects to initially deleted.
Since you have linked to this question from my own question, I thought I'd throw in some newly-aquired experience with Entity Framework for me.
To achieve a common save method in my generic repository with Entity Framework, I do this. (Please note that the Context is a member of my repository, as I am implementing the Unit of Work pattern as well)
public class EFRepository<TEntity> : IRepository<TEntity> where TEntity : class
{
internal readonly AwesomeContext Context;
internal readonly DbSet<TEntity> DbSet;
public EFRepository(AwesomeContext context)
{
if (context == null) throw new ArgumentNullException("context");
Context = context;
DbSet = context.Set<TEntity>();
}
// Rest of implementation removed for brevity
public void Save(TEntity entity)
{
var entry = Context.Entry(entity);
if (entry.State == EntityState.Detached)
DbSet.Add(entity);
else entry.State = EntityState.Modified;
}
}
Honestly, I can't tell you why this works, because I just kept changing the state conditions - however I do have unit (integration) tests to prove that it works. Hopefully someone more into EF than myself can shed some light on this.
Regarding the "cascading updates", I was curious myself as if it would work using the Unit of Work pattern (my question I linked to was when I did not know it existed, and my repositories would basically create a unit of work whenever I wanted to save/get/delete, which is bad), so I threw in a test case in a simple relational DB. Here is a diagram to give you an idea.
IMPORTANT In order for test case number 2 to work, you need to make your POCO reference properties virtual, in order for EF to provide lazy loading.
The repository implementation is just derived from the generic EFRepository<TEntity> as shown above, so I'll leave out that implementation.
These are my test cases, both pass.
public class EFResourceGroupFacts
{
[Fact]
public void Saving_new_resource_will_cascade_properly()
{
// Recreate a fresh database and add some dummy data.
SetupTestCase();
using (var ctx = new LocalizationContext("Localization.CascadeTest"))
{
var cultureRepo = new EFCultureRepository(ctx);
var resourceRepo = new EFResourceRepository(cultureRepo, ctx);
var existingCulture = cultureRepo.Get(1); // First and only culture.
var groupToAdd = new ResourceGroup("Added Group");
var resourceToAdd = new Resource(existingCulture,"New Resource", "Resource to add to existing group.",groupToAdd);
// Verify we got a single resource group.
Assert.Equal(1,ctx.ResourceGroups.Count());
// Saving the resource should also add the group.
resourceRepo.Save(resourceToAdd);
ctx.SaveChanges();
// Verify the group was added without explicitly saving it.
Assert.Equal(2, ctx.ResourceGroups.Count());
}
// try creating a new Unit of Work to really verify it has been persisted..
using (var ctx = new LocalizationContext("Localization.CascadeTest"))
{
Assert.DoesNotThrow(() => ctx.ResourceGroups.First(rg => rg.Name == "Added Group"));
}
}
[Fact]
public void Changing_existing_resources_group_saves_properly()
{
SetupTestCase();
using (var ctx = new LocalizationContext("Localization.CascadeTest"))
{
ctx.Configuration.LazyLoadingEnabled = true;
var cultureRepo = new EFCultureRepository(ctx);
var resourceRepo = new EFResourceRepository(cultureRepo, ctx);
// This resource already has a group.
var existingResource = resourceRepo.Get(2);
Assert.NotNull(existingResource.ResourceGroup); // IMPORTANT: Property must be virtual!
// Verify there is only one resource group in the datastore.
Assert.Equal(1,ctx.ResourceGroups.Count());
existingResource.ResourceGroup = new ResourceGroup("I am implicitly added to the database. How cool is that?");
// Make sure there are 2 resources in the datastore before saving.
Assert.Equal(2, ctx.Resources.Count());
resourceRepo.Save(existingResource);
ctx.SaveChanges();
// Make sure there are STILL only 2 resources in the datastore AFTER saving.
Assert.Equal(2, ctx.Resources.Count());
// Make sure the new group was added.
Assert.Equal(2,ctx.ResourceGroups.Count());
// Refetch from store, verify relationship.
existingResource = resourceRepo.Get(2);
Assert.Equal(2,existingResource.ResourceGroup.Id);
// let's change the group to an existing group
existingResource.ResourceGroup = ctx.ResourceGroups.First();
resourceRepo.Save(existingResource);
ctx.SaveChanges();
// Assert no change in groups.
Assert.Equal(2, ctx.ResourceGroups.Count());
// Refetch from store, verify relationship.
existingResource = resourceRepo.Get(2);
Assert.Equal(1, existingResource.ResourceGroup.Id);
}
}
private void SetupTestCase()
{
// Delete everything first. Database.SetInitializer does not work very well for me.
using (var ctx = new LocalizationContext("Localization.CascadeTest"))
{
ctx.Database.Delete();
ctx.Database.Create();
var culture = new Culture("en-US", "English");
var resourceGroup = new ResourceGroup("Existing Group");
var resource = new Resource(culture, "Existing Resource 1",
"This resource will already exist when starting the test. Initially it has no group.");
var resourceWithGroup = new Resource(culture, "Exising Resource 2",
"Same for this resource, except it has a group.",resourceGroup);
ctx.Cultures.Add(culture);
ctx.ResourceGroups.Add(resourceGroup);
ctx.Resources.Add(resource);
ctx.Resources.Add(resourceWithGroup);
ctx.SaveChanges();
}
}
}
It was interesting to learn this, as I was not sure if it would work.
After working on this for a while I found an opensource project called GraphDiff here is it's blog entry 'introducing graphdiff for entity framework code first – allowing automated updates of a graph of detached entities'. I only began using it but it looks impressive. And it does solve the problem of issuing update/delete/insert for Many to One relationships. It actually generalizes the problem to graphs and allows arbitrary nesting.
Here is the generic method I concocted. It does use AddOrUpdate from the System.Data.Entity.Migrations namespace. Which may be reloading records from the db, I'll be checking on that later. The usage is
ctx.OrderLines.AddOrUpdateSet(l => l.orderId == neworder.Id,
l => l.Id, order.orderLines);
Here is the code:
public static class UpdateExtensions
{
public static void AddOrUpdateSet<TEntity>(this IDbSet<TEntity> set, Expression<Func<TEntity, bool>> predicate,
Func<TEntity, int> selector, IEnumerable<TEntity> newRecords) where TEntity : class
{
List<TEntity> oldRecords = set.Where(predicate).ToList();
IEnumerable<int> keys = newRecords.Select(selector);
foreach (TEntity newRec in newRecords)
set.AddOrUpdate(newRec);
oldRecords.FindAll(old => !keys.Contains(selector(old))).ForEach(detail => set.Remove(detail));
}
}
I have a FunctionalTest that tests posting to a controller and then does asserts on the model objects to make sure the controller did it's job, like so:
#Test
public void editUser(){
Logger.debug("Edit user test");
createNewUser();
final User user = User.<User>findAll().get(0);
POST("/ManageUser/save", ImmutableMap.of(
"user.id", user.getId().toString(),
"user.username", "test",
"user.email", "test#example.com",
"user.fullName", "Test Different"
));
User.em().flush();
User.em().clear(); // this is required so that it works on the mem DB
assertEquals(1, User.findAll().size());
assertEquals("Test Different", User.<User>findAll().get(0).fullName);
final User userAfterSave = User.<User>findAll().get(0);
assertFalse("New user should not be admin.", userAfterSave.isAdmin);
}
This passes when I use the mem database
%test.db.url=jdbc:h2:mem:play;MODE=MYSQL;LOCK_MODE=0
However if I switch to mysql
%test.db=mysql://test:test#localhost/test
It fails on the second assert "Failure, expected:<Test [Differen]t> but was:<Test [Tes]t>". So when using mysql the controller doesn't persist the user properly.
What am I missing here, are there some options for transaction control that I need to change for this to work?
The controller just calls user.merge().save() to update the user, is this somehow wrong?
This is because the mem DB is not properly transactional, meaning the test thread gets new data every read. For mysql however the test thread read the user in createNewUser() meaning it's transaction had the previous version. It's not obvious but POST() starts a new thread with a separate transaction. To solve this swap out
User.em().flush();
User.em().clear();
for
JPAPlugin.closeTx(false);
JPAPlugin.startTx(false);
The later starts a new transaction.
Do your search in a separate job to be sure it correctly view the modifications (transaction isolation). Here is an example
private FeedbackType findFeedbackType(final String name) throws ExecutionException, InterruptedException {
return new Job<FeedbackType>() {
#Override
public FeedbackType doJobWithResult() throws Exception {
return FeedbackType.findByName(name);
}
}.now().get();
}
This is a private method of my Functional test and I call this method to get my object instead of directly invoking the model