I recently encountered a situation, where it was possible to trigger a database update via a REST endpoint, but not via a #Scheduled method:
#Service
class DbTestService(
private val repo: TestRepository,
private val tx: TransactionTemplate,
private val em: EntityManager,
) {
fun updateAllEntities() {
val entities = repo.findAll() // intentionally outside transaction
tx.execute {
for (entity in entities) {
// if called from SchedulingService, this prints false
// if called from TestController, this prints true
println(" entity named '${entity.name}' is managed: ${em.contains(entity)}")
entity.lastChanged = LocalDateTime.now()
}
}
}
}
#Service
class SchedulingService(
private val dbService: DbTestService,
) {
#Scheduled(cron = "0/20 * * * * *")
fun scheduledUpdate() {
dbService.updateAllEntities() // DOES NOT persist the update
}
}
#RestController
class TestController(
private val dbService: DbTestService,
) {
#GetMapping("/updateAll")
fun updateAll() {
dbService.updateAllEntities() // DOES persist the update
}
}
While I know how to fix this, (just move the repo.findAll() inside the transaction) I find it rather curious that this is the actual behaviour of the system. Scheduled methods seem to run in a different persistence context, but I currently lack the detailed knowledge to fully understand the situation.
Can someone please explain, why this is happening exactly?
I also created a minimal working example project to demonstrate the situation.
Related
I'm kind of new to reactive programing and currently working on a spring webflux based application. I'm stuck between few questions.
public class FooServiceImpl {
#Autowired
private FooDao fooDao;
#Autowired
private AService aService;
#Autowired
private BService bService;
public long calculateSomething(long fooId) {
Foo foo = fooDao.findById(fooId); // Blocking call one
if (foo == null) {
foo = new Foo();
}
Long bCount = bService.getCountBByFooId(fooId); // Blocking call two
AEntity aEntity = aService.getAByFooId(fooId); // Blocking call three
// Do some calculation using foo, bCount and aEntity
// ...
// ...
return someResult;
}
}
This is the way we write a blocking code which uses three external API call results (let's consider as DB calls). I'm struggling to convert this into a reactive code, If all three becomes mono and if I subscribe all three will the outer subscriber get blocked?
public Mono<Long> calculateSomething(long fooId) {
return Mono.create(sink -> {
Mono<Foo> monoFoo = fooDao.findById(fooId); // Reactive call one
monoFoo.subscribe(foo -> {
if (foo == null) {
foo = new Foo();
}
Mono<Long> monoCount = bService.getCountBByFooId(fooId); // Reactive call two
monoCount.subscribe(aLong -> {
Mono<AEntity> monoA = aService.getAByFooId(fooId); // Reactive call three
monoA.subscribe(aEntity -> {
//...
//...
sink.success(someResult);
});
});
});
};
}
I saw there is a function called zip, but it only works with two results, So is there a way to apply it here?
Also what will happen if we get subscribe for something inside create method, Will it block the thread?
Would be very thankful if you could help me.
If you gave me the calculation you want you do with those values, it would be easier for me to show the reactor way of doing it. But lets suppose you want to read a value from database and then use that value for another thing. Use flatmaps and make a unique Flux reducing the lines of code and complexity, no need to use subscribe() as told by the other people. Example:
return fooDao.findById(fooId)
.flatmap(foo -> bService.getCountBByFooId(foo))
.flatmap(bCount -> aService.getAByFooId(fooId).getCount()+bCount);
I have a problem with isolation levels in JPA. For example I have following code:
#Transactional(isolation = Isolation.SERIALIZABLE)
public void first() {
Obj obj = new Obj();
obj.setName("t");
objDAO.save(obj);
second();
}
#Transactional(propagation = Propagation.REQUIRES_NEW, isolation = Isolation.SERIALIZABLE)
public void second(){
List<Obj> objs = objDAO.findAll();
}
In my opinion the second method should not see uncomitted changes from method first. So new object with name "t" should not be visible till commit (but it is).
If I am wrong, then please, give me example in JPA where it won't be visible. Many thanks for any advice.
If your methods are inside one class it will not work because container will treat this as a single transaction. The container doesn't know that you want to create new transaction.
From the Spring reference:
Note: In proxy mode (which is the default), only 'external' method calls coming in through the proxy will be intercepted. This means that 'self-invocation', i.e. a method within the target object calling some other method of the target object, won't lead to an actual transaction at runtime even if the invoked method is marked with #Transactional!
If you want to call method second() in new transaction you can try this:
#Autowired
private ApplicationContext applicationContext;
#Transactional(isolation = Isolation.SERIALIZABLE)
public void first() {
Obj obj = new Obj();
obj.setName("t");
objDAO.save(obj);
applicationContext.getBean(getClass()).second();
}
#Transactional(propagation = Propagation.REQUIRES_NEW, isolation = Isolation.SERIALIZABLE)
public void second(){
List<Obj> objs = objDAO.findAll();
}
I have a Object like this:
// I want to test this Object
object MyObject {
protected val retryHandler: HttpRequestRetryHandler = new HttpRequestRetryHandler {
def retryRequest(exception: IOException, executionCount: Int, context: HttpContext): Boolean = {
true // implementation
}
}
private val connectionManager: PoolingHttpClientConnectionManager = new PoolingHttpClientConnectionManager
val httpClient: CloseableHttpClient = HttpClients.custom
.setConnectionManager(connectionManager)
.setRetryHandler(retryHandler)
.build
def methodPost = {
//create new context and new Post instance
val post = new HttpPost("url")
val res = httpClient.execute(post, HttpClientContext.create)
// check response code and then take action based on response code
}
def methodPut = {
// same as methodPost except use HttpPut instead HttpPost
}
}
I want to test this object by mocking dependent objects like httpClient. How to achieve this? can i do it using Mokito or any better way? If yes. How? Is there a better design for this class?
Your problem is: you created hard-to test code. You can turn here to watch some videos to understand why that is.
The short answer: directly calling new in your production code always makes testing harder. You could be using Mockito spies (see here on how that works).
But: the better answer would be to rework your production code; for example to use dependency injection. Meaning: instead of creating the objects your class needs itself (by using new) ... your class receives those objects from somewhere.
The typical (java) approach would be something like:
public MyClass() { this ( new SomethingINeed() ); }
MyClass(SomethingINeed incoming) { this.somethign = incoming; }
In other words: the normal usage path still calls new directly; but for unit testing you provide an alternative constructor that you can use to inject the thing(s) your class under test depends on.
I have a base object, that contains a Version property, marked as ConcurrencyCheck
public class EntityBase : IEntity, IConcurrencyEnabled
{
public int Id { get; set; }
[ConcurrencyCheck]
[Timestamp]
public byte[] Version { get; set; }
}
This works, however, I want to write a test to ensure it doesn't get broken. Unfortunately, I can't seem to figure out how to write a test that doesn't rely on the physical database!
And the relevant test code that works, but uses the database...
protected override void Arrange()
{
const string asUser = "ConcurrencyTest1"; // used to anchor and lookup this test record in the db
Context1 = new MyDbContext();
Context2 = new MyDbContext();
Repository1 = new Repository<FooBar>(Context1);
Repository2 = new Repository<FooBar>(Context2);
UnitOfWork1 = new UnitOfWork(Context1);
UnitOfWork2 = new UnitOfWork(Context2);
Sut = Repository1.Find(x => x.CreatedBy.Equals(asUser)).FirstOrDefault();
if (Sut == null)
{
Sut = new FooBar
{
Name = "Concurrency Test"
};
Repository1.Insert(Sut);
UnitOfWork1.SaveChanges(asUser);
}
ItemId = Sut.Id;
}
protected override void Act()
{
_action = () =>
{
var item1 = Repository1.FindById(ItemId);
var item2 = Repository2.FindById(ItemId);
item1.Name = string.Format("Changed # {0}", DateTime.Now);
UnitOfWork1.SaveChanges("test1");
item2.Name = string.Format("Conflicting Change # {0}", DateTime.Now);
UnitOfWork2.SaveChanges("test2"); //Should throw DbUpdateConcurrencyException
};
}
[TestMethod]
[ExpectedException(typeof(DbUpdateConcurrencyException))]
public void Assert()
{
_action();
}
How can I remove the DB requirement???
I would recommend extracting your MyDbContext into an interface IMyDbContext, and then creating a TestDbContext class that will also implement SaveChanges the way you have it up there, except with returning a random value (like 1) instead of actually saving to the database.
At that point then all you'd need to do is to test that, in fact, all of the entities got their version number upped.
Or you could also do the examples found here or here, as well.
EDIT: I actually just found a direct example with using TimeStamp for concurrency checks on this blog post.
It's my opinion that you should not try to mock this behaviour to enable "pure" unit testing. For two reasons:
it requires quite a lot of code that mocks database behaviour: materializing objects in a way that they have a version value, caching the original objects (to mock a store), modifying the version value when updating, comparing the version values with the original ones, throwing an exception when a version is different, and maybe more. All this code is potentially subject to bugs and, worse, may differ slightly from what happens in reality.
you'll get trapped in circular reasoning: you write code specifically for unit tests and then... you write unit tests to test this code. Green tests say everything is OK, but essential parts of application code are not covered.
This is only one of the many aspects of linq to entities that are hard (impossible) to mock. I am compiling a list of these differences here.
I have read dozens of posts about PROs and CONs of trying to mock \ fake EF in the business logic.
I have not yet decided what to do - but one thing I know is - I have to separate the queries from the business logic.
In this post I saw that Ladislav has answered that there are 2 good ways:
Let them be where they are and use custom extension methods, query views, mapped database views or custom defining queries to define reusable parts.
Expose every single query as method on some separate class. The method
mustn't expose IQueryable and mustn't accept Expression as parameter =
whole query logic must be wrapped in the method. But this will make
your class covering related methods much like repository (the only one
which can be mocked or faked). This implementation is close to
implementation used with stored procedures.
Which method do you think is better any why ?
Are there ANY downsides to put the queries in their own place ? (maybe losing some functionality from EF or something like that)
Do I have to encapsulate even the simplest queries like:
using (MyDbContext entities = new MyDbContext)
{
User user = entities.Users.Find(userId); // ENCAPSULATE THIS ?
// Some BL Code here
}
So I guess your main point is testability of your code, isn't it? In such case you should start by counting responsibilities of the method you want to test and than refactor your code using single responsibility pattern.
Your example code has at least three responsibilities:
Creating an object is a responsibility - context is an object. Moreover it is and object you don't want to use in your unit test so you must move its creation elsewhere.
Executing query is a responsibility. Moreover it is a responsibility you would like to avoid in your unit test.
Doing some business logic is a responsibility
To simplify testing you should refactor your code and divide those responsibilities to separate methods.
public class MyBLClass()
{
public void MyBLMethod(int userId)
{
using (IMyContext entities = GetContext())
{
User user = GetUserFromDb(entities, userId);
// Some BL Code here
}
}
protected virtual IMyContext GetContext()
{
return new MyDbContext();
}
protected virtual User GetUserFromDb(IMyDbContext entities, int userId)
{
return entities.Users.Find(userId);
}
}
Now unit testing business logic should be piece of cake because your unit test can inherit your class and fake context factory method and query execution method and become fully independent on EF.
// NUnit unit test
[TestFixture]
public class MyBLClassTest : MyBLClass
{
private class FakeContext : IMyContext
{
// Create just empty implementation of context interface
}
private User _testUser;
[Test]
public void MyBLMethod_DoSomething()
{
// Test setup
int id = 10;
_testUser = new User
{
Id = id,
// rest is your expected test data - that is what faking is about
// faked method returns simply data your test method expects
};
// Execution of method under test
MyBLMethod(id);
// Test validation
// Assert something you expect to happen on _testUser instance
// inside MyBLMethod
}
protected override IMyContext GetContext()
{
return new FakeContext();
}
protected override User GetUserFromDb(IMyContext context, int userId)
{
return _testUser.Id == userId ? _testUser : null;
}
}
As you add more methods and your application grows you will refactor those query execution methods and context factory method to separate classes to follow single responsibility on classes as well - you will get context factory and either some query provider or in some cases repository (but that repository will never return IQueryable or get Expression as parameter in any of its methods). This will also allow you following DRY principle where your context creation and most commonly used queries will be defined only once on one central place.
So at the end you can have something like this:
public class MyBLClass()
{
private IContextFactory _contextFactory;
private IUserQueryProvider _userProvider;
public MyBLClass(IContextFactory contextFactory, IUserQueryProvider userProvider)
{
_contextFactory = contextFactory;
_userProvider = userProvider;
}
public void MyBLMethod(int userId)
{
using (IMyContext entities = _contextFactory.GetContext())
{
User user = _userProvider.GetSingle(entities, userId);
// Some BL Code here
}
}
}
Where those interfaces will look like:
public interface IContextFactory
{
IMyContext GetContext();
}
public class MyContextFactory : IContextFactory
{
public IMyContext GetContext()
{
// Here belongs any logic necessary to create context
// If you for example want to cache context per HTTP request
// you can implement logic here.
return new MyDbContext();
}
}
and
public interface IUserQueryProvider
{
User GetUser(int userId);
// Any other reusable queries for user entities
// Non of queries returns IQueryable or accepts Expression as parameter
// For example: IEnumerable<User> GetActiveUsers();
}
public class MyUserQueryProvider : IUserQueryProvider
{
public User GetUser(IMyContext context, int userId)
{
return context.Users.Find(userId);
}
// Implementation of other queries
// Only inside query implementations you can use extension methods on IQueryable
}
Your test will now only use fakes for context factory and query provider.
// NUnit + Moq unit test
[TestFixture]
public class MyBLClassTest
{
private class FakeContext : IMyContext
{
// Create just empty implementation of context interface
}
[Test]
public void MyBLMethod_DoSomething()
{
// Test setup
int id = 10;
var user = new User
{
Id = id,
// rest is your expected test data - that is what faking is about
// faked method returns simply data your test method expects
};
var contextFactory = new Mock<IContextFactory>();
contextFactory.Setup(f => f.GetContext()).Returns(new FakeContext());
var queryProvider = new Mock<IUserQueryProvider>();
queryProvider.Setup(f => f.GetUser(It.IsAny<IContextFactory>(), id)).Returns(user);
// Execution of method under test
var myBLClass = new MyBLClass(contextFactory.Object, queryProvider.Object);
myBLClass.MyBLMethod(id);
// Test validation
// Assert something you expect to happen on user instance
// inside MyBLMethod
}
}
It would be little bit different in case of repository which should have reference to context passed to its constructor prior to injecting it to your business class.
Your business class can still define some queries which are never use in any other classes - those queries are most probably part of its logic. You can also use extension methods to define some reusable part of queries but you must always use those extension methods outside of your core business logic which you want to unit test (either in query execution methods or in query provider / repository). That will allow you easy faking query provider or query execution methods.
I saw your previous question and thought about writing a blog post about that topic but the core of my opinion about testing with EF is in this answer.
Edit:
Repository is different topic which doesn't relate to your original question. Specific repository is still valid pattern. We are not against repositories, we are against generic repositories because they don't provide any additional features and don't solve any problem.
The problem is that repository alone doesn't solve anything. There are three patterns which have to be used together to form proper abstraction: Repository, Unit of Work and Specifications. All three are already available in EF: DbSet / ObjectSet as repositories, DbContext / ObjectContext as Unit of works and Linq to Entities as specifications. The main problem with custom implementation of generic repositories mentioned everywhere is that they replace only repository and unit of work with custom implementation but still depend on original specifications => abstraction is incomplete and it is leaking in tests where faked repository behaves in the same way as faked set / context.
The main disadvantage of my query provider is explicit method for any query you will need to execute. In case of repository you will not have such methods you will have just few methods accepting specification (but again those specifications should be defined in DRY principle) which will build query filtering conditions, ordering etc.
public interface IUserRepository
{
User Find(int userId);
IEnumerable<User> FindAll(ISpecification spec);
}
The discussion of this topic is far beyond the scope of this question and it requires you to do some self study.
Btw. mocking and faking has different purpose - you fake a call if you need to get testing data from method in the dependency and you mock the call if you need to assert that method on dependency was called with expected arguments.