I'm working on a code base has lots of singleton daos which getting a new connection from pool in each method with DB.withConnection then executes block.
Dao methods use Anorm parsers to parse result set. There are some cases each dao method run some other dao methods in anorm parsers to get nested related items for business model.
Let's assume you have data structure like this;
User -> Posts -> Comments for each posts
Dao work like this;
UserDao.getUser
PostDao.getUserPosts
CommentsDao.getPostComments
Because each dao method is calling DB.withConnection, multiple connections used for simple operations.
I want to use same connection. It can be done with implicit connection passing through each dao method.
But then I need to maintain connection allocation in upper layer. Right now daos accessed directly from rest actions, there is not any kind of service layer sitting between api and dao. I feel like it's not good to have connection around just in api layer.
So probably having services like UserService which calling daos and handling connection and transaction would be better.
Then another requirement is make me uncomfortable. Most of the dao methods also need to be called by individually.
For example we have api requesting comments only by;
CommentsDao.getPostComments
This means also I need to implement Services for all daos basically, overriding each dao methods with DB.withConnection wrapper, seems like a overhead. (~30 daos, ~10 methods)
Another limitation is that actually inner dao calls is done in anorm parsers, I think this one is misuse of the library;
When we change the each dao method definition to pass implicit connection around, parsers would fail to compile.
Because anorm parsers is ResultSetParser[T] , there is no way to pass connection inside it as default. Example parser is like;
val apiProviderParser = get[Int]("id") ~
get[String]("product_name") ~
get[String]("description") ~
get[String]("icon_url") map {
case id ~ productName ~ desc ~ iconUrl => {
//Inner dao call
//Connection needed now
val params = getApiProviderParams(id)//(implicit connection)
new ApiProviderTemplate(id,productName,desc,iconUrl,params)
}
}
Maybe custom ResultSetParser[T] with connection in the scope can work, I'm not sure it's correct way of solving this problem.
I checked cake pattern, dao design issues but couldn't decide how to proceed and what's the pragmatic, good solution for this problem.
Any help appreciated.
Related
I have an actor (actually a persistent actor) that, in response to a message (command), needs to make an HTTP call. The question now is how do I deal with this HTTP call in unit test?
Normally I would use the combination of DI and mocking to inject a mock implementation of the HTTP service while testing. But I am not sure if this is how to approach the problem in Akka? Even if this is the how to approach it in Akka, I am not sure how to go about doing the injecting and mocking in testing.
Any thoughts on this? What is the idiomatic way for testing Actors that perform IO operations (HTTP calls, writing to DB etc).
PS: I am using Akka Typed.
My personal belief is that you should avoid IO operations in Actors if at all possible (see this presentation for more details).
That being said, I’m sure there are people who would disagree and you probably shouldn’t listen to me :)
Here is how I would go about mocking it for a test.
Create a trait that represents your http call.
trait Client {
def makeHttpCall(input: Int): Future[Output]
}
(You will want to create a class of some kind that implements this trait and test it separately from your actor.)
Pass an instance of that trait into your actor using its constructor/apply method.
def apply(client: Client): Behavior[...] = {
// use your client inside of your behavior
}
In your test create a mock instance of Client and pass that in to your actor. You can mock your Client using a mocking library (ScalaMock or Mockito) or you can actually just create a test implementation of your trait with relative ease:
class TestClient extends Client {
def makeHttpCall(input: Int): Future[Output] =
Future.successful(Output(...))
}
Note: all of the names for classes and methods that I chose are just placeholders. You should, of course, choose more specific names based on the context you are working in.
I am creating a repository in my project which will be responsible for all storage operations for a User entity. I will use mongo as the db and mongoreactive as the client. The problem which I have now is about types.
trait UserRepository {
save(user: User) : ?
}
trait MongoUserRepository extends UserRepository {
save(user: User) : Future[WriteResult] = {
collection.insert(user)
}
}
How in my domain should I model WriteResult which comes from MongoReactive? I do not want it to leak into my domain. Is there any existing pattern or a good practice?
How in my domain should I model WriteResult which comes from MongoReactive? I do not want it to leak into my domain. Is there any existing pattern or a good practice?
The usual practice is that the domain would define the UserRepository trait as a service provider interface (spi) that the persistence infrastructure would need to support. Fundamentally, it's a way of expressing the usage requirements that the model imposes on persistence.
Using the language of Command Query Separation, save is a command: it's an operation that changes the state of the repository. So the implementation of the trait should conform to your local coding standard for implementing a command.
Greg Young on (generic) repositories:
What exactly was the intent of the repository pattern in the first place? Looking back to [DDD, Evans] one will see that it is to represent a series of objects as if they were a collection in memory so that the domain can be freed of persistence concerns. In other words the goal is to put collection semantics on the objects in persistence.
So you could also to your collections library for inspiration.
But generally speaking, the most common choice would look like
trait UserRepository {
save(user: User) : Unit
}
So that is the contract your specific implementations would be expected to satisfy.
In MongoUserRepository, you adapt the implementation of your persistence solution to satisfy the contract. In this case that would mean unpacking the Future, inspecting the WriteResult for errors, and throwing an exception if the write was unsuccessful.
With save(user: User) : Unit you implicitly put a requirement on your clients to watch for repository failure (for example: in case of db failure)
Other way around - the repository is a service provider interface; this design doesn't constrain the clients, but the providers. In the lingo of hexagonal architecture, I'm defining a secondary port and constraining the secondary adapter to conform to the contract of the port.
The motivation is exactly the one you describe: the repository consumer is supposed to be isolated from the protocol required to interact with the selected persistence solution. The domain model sits in the middle of the business universe, and the adapter insulates the business universe from reality.
Evans Chapter 6 raises the management challenge of "preventing the model from getting swamped by the complexity of managing the (domain object) life cycle". Repositories provide "the means of finding and retrieving persistent objects while encapsulating the immense infrastructure involved.
The repository is a firewall.
What we are addressing here is separation of concerns; the domain model describes your business logic. We want to be able to see the business case clearly, and that's not going to be possible if we have to explicitly manage what happens in the event that a mutable, in memory, collection fails catastrophically on modification. The pragmatic answer is to nope out and let the application handle it.
That said... I deliberately hedged above when I wrote "conform to your local coding standard". If your local guidelines use railway oriented programming, or message driven, then by all means line it up. There's absolutely no reason why the domain model should care whether storage is synchronous or asynchronous, local or remote, etc.
But if your domain model starts to fill up with match expressions that describe implementation concerns, I'd say you've lost the plot somewhere.
When I was implementing the same repository, I ended up extracting WriteResult value I was most interested in. In my case I ended up with the following signature:
trait UserRepository {
save(user: User) : Future[Option[String]]
}
which returns either some error message or nothing. As a result implementation will be like this:
trait MongoUserRepository extends UserRepository {
save(user: User) : Future[Option[String]] = {
collection.insert(user).map(_.errmsg)
}
}
I ended up with this implementation because I do not lose exception message in case of exception.
Alternative options could be to map insertion result to Boolean:
trait UserRepository {
save(user: User) : Future[Boolean]
}
trait MongoUserRepository extends UserRepository {
save(user: User) : Future[Boolean] = {
collection.insert(user).map(_.ok)
}
}
But in this case you will lose exception message. Sometimes it can be fine however it depends on your exact case.
UPDATE: The answer posted above is valid for 0.11 version. In 0.12 method errmsg in WriteResult was removed. Alternatively you can use writeErrors and in case Seq is not empty you extract all errmsgs from all WriteError.
Hope it helps, sir!
I am doing a web application with Scala and Akka actors and I'm having some troubles with the tests.
In my case I need to taste an actor who talks with the Database. To do the unit testing I would like to use a Fake Database but I can't replace the new with my desired fake object.
Let's see some code:
Class MyActor extends Actor {
val database = new Database()
def receive = { ... }
}
And in the tests I would like to inject a FakeDatabase object instead Database. I've been looking in Internet but the best that I found is:
Add a parameter to the constructor.
Convert the val database to a var so in the test I could access the attribute by the underlying and replace it.
Both solutions solve the problem but are very dirty.
Isn't a better way to solve the problem?
Thanks!
The two primary options for this scenario are:
Dependency Injection Use a DI framework to inject a real or mock service as needed. In Akka: http://letitcrash.com/post/55958814293/akka-dependency-injection
Cake Pattern This is a Scala-specific way of achieving something akin to dependency injection without actually relying on injection. See: Akka and cake pattern
Echoing the advice here, I wouldn't call injecting the database in the constructor dirty. It might have plenty of benefits, including decoupling actor behaviour from the particular database instance.
However if you know there is only ONE database you will be always using in your production code, then think about defining a package level accessible constructor and a companion object returning a Props object without parameters by default.
Example below:
object MyActor {
def props() : Props = Props(new MyActor(new Database()))
}
class MyActor private[package](database : IDatabase) extends Actor {
def receive = { ... }
}
In this case you will still be able to inject the test database in your tests case (given the same package structure), but prevent users of your code from instantiating MyActor with unexpected database instance.
UPDATE:
I've edited the title and added this text to better explain what I'm trying to achieve: I'm trying to create a new application from the ground up, but don't want the business layer to know about the persistence layer, in the same way one would not want the business layer to know about a REST API layer. Below is an example of a persistence layer that I would like to use. I'm looking for good advice on integrating with this i.e. I need help with the design/architecture to cleanly split the responsibilities between business logic and persistence logic. Maybe a concept along the line of marshalling and unmarshalling of persistence objects to domain objects.
From a SLICK (a.k.a. ScalaQuery) test example, this is how you create a many-to-many database relationship. This will create 3 tables: a, b and a_to_b, where a_to_b keeps links of rows in table a and b.
object A extends Table[(Int, String)]("a") {
def id = column[Int]("id", O.PrimaryKey)
def s = column[String]("s")
def * = id ~ s
def bs = AToB.filter(_.aId === id).flatMap(_.bFK)
}
object B extends Table[(Int, String)]("b") {
def id = column[Int]("id", O.PrimaryKey)
def s = column[String]("s")
def * = id ~ s
def as = AToB.filter(_.bId === id).flatMap(_.aFK)
}
object AToB extends Table[(Int, Int)]("a_to_b") {
def aId = column[Int]("a")
def bId = column[Int]("b")
def * = aId ~ bId
def aFK = foreignKey("a_fk", aId, A)(a => a.id)
def bFK = foreignKey("b_fk", bId, B)(b => b.id)
}
(A.ddl ++ B.ddl ++ AToB.ddl).create
A.insertAll(1 -> "a", 2 -> "b", 3 -> "c")
B.insertAll(1 -> "x", 2 -> "y", 3 -> "z")
AToB.insertAll(1 -> 1, 1 -> 2, 2 -> 2, 2 -> 3)
val q1 = for {
a <- A if a.id >= 2
b <- a.bs
} yield (a.s, b.s)
q1.foreach(x => println(" "+x))
assertEquals(Set(("b","y"), ("b","z")), q1.list.toSet)
As my next step, I would like to take this up one level (I still want to use SLICK but wrap it nicely), to working with objects. So in pseudo code it would be great to do something like:
objectOfTypeA.save()
objectOfTypeB.save()
linkAtoB.save(ojectOfTypeA, objectOfTypeB)
Or, something like that. I have my ideas on how I might approach this in Java, but I'm starting to realize that some of my object-oriented ideas from pure OO languages are starting to fail me. Can anyone please give me some pointers as to how approach this problem in Scala.
For example: Do I create simple objects that just wrap or extend the table objects, and then include these (composition) into another class that manages them?
Any ideas, guidance, example (please), that will help me better approach this problem as a designer and coder will be greatly appreciated.
The best idea would be to implement something like data mapper pattern. Which, in contrast to active record, will not violate SRP.
Since I am not a Scala developer, I will not show any code.
The idea is following:
create domain object instance
set conditions on the element (for example setId(42), if you are looking for element by ID)
create data mapper instance
execute fetch() method on the mapper by passing in domain object as parameter
The mapper would look up current parameters of provided domain object and, based on those parameters, retrieve information from storage (which might be SQL database, or JSON file or maybe a remote REST API). If information is retrieved, it assigns the values to the domain object.
Also, I must note, that data mappers are created for work with specific domain object's interface, but the information, which they pass from domain object to storage and back, can be mapped to multiple SQL tables or multiple REST resources.
This way you can easily replace the mapper, when you switch to different storage medium, or even unit-test the logic in domain objects without touching the real storage. Also, if you decide to add caching at some point, that would be just another mapper, which tried to fetch information from cache, and, if it fails, the mapper for persistent storage kicks in.
Domain object (or, in some cases, a collection of domain objects) would be completely unaware of whether it is stored or retrieved. That would be the responsibility of the data mappers.
If this is all in MVC context, then, to fully implement this, you would need another group of structures in the model layer. I call them "services" (please share, of you come up with better name). They are responsible for containing the interaction between data mappers and domain objects. This way you can prevent the business logic from leaking in the presentation layer (controllers, to be exact), and these services create a natural interface for interaction between business (also know as model) layer and the presentation layer.
P.S. Once again, sorry that I cannot provide any code examples, because I am a PHP developer and have no idea how to write code in Scala.
P.P.S. If you are using data mapper pattern, the best option is to write mappers manually and not use any 3rd party ORM, which claims to implement it. It would give you more control over codebase and avoid pointless technical debt [1] [2].
A good solution for simple persistence requirements is the ActiveRecord pattern: http://en.wikipedia.org/wiki/Active_record_pattern . This is implemented in Ruby and in Play! framework 1.2, and you can easily implement it in Scala in a stand-alone application
The only requirement is to have a singleton DB or a singleton service to get a reference to the DB you require. I personally would go for an implementation based on the following:
A generic trait ActiveRecord
A generic typeclass ActiveRecordHandler
Exploiting the power of implicits, you could obtain an amazing syntax:
trait ActiveRecordHandler[T]{
def save(t:T):T
def delete[A<:Serializable](primaryKey:A):Option[T]
def find(query:String):Traversable[T]
}
object ActiveRecordHandler {
// Note that an implicit val inside an object with the same name as the trait
// is one of the way to have the implicit in scope.
implicit val myClassHandler = new ActiveRecordHandler[MyClass] {
def save(myClass:MyClass) = myClass
def delete[A <: Serializable](primaryKey: A) = None
def find(query: String) = List(MyClass("hello"),MyClass("goodbye"))
}
}
trait ActiveRecord[RecordType] {
self:RecordType=>
def save(implicit activeRecordHandler:ActiveRecordHandler[RecordType]):RecordType = activeRecordHandler.save(this)
def delete[A<:Serializable](primaryKey:A)(implicit activeRecordHandler:ActiveRecordHandler[RecordType]):Option[RecordType] = activeRecordHandler.delete(primaryKey)
}
case class MyClass(name:String) extends ActiveRecord[MyClass]
object MyClass {
def main(args:Array[String]) = {
MyClass("10").save
}
}
With such a solution, you only need your class to extends ActiveRecord[T] and have an implicit ActiveRecordHandler[T] to handle this.
There is actually also an implementation: https://github.com/aselab/scala-activerecord which is based on similar idea, but instead of making the ActiveRecord having an abstract type, it declares a generic companion object.
A general but very important comment on the ActiveRecord pattern is that it helps meet simple requirements in terms of persistence, but cannot deal with more complex requirements: for example is when you want to persist multiple objects under the same transaction.
If your application requires more complex persistence logic, the best approach is to introduce a persistence service which exposes only a limited set of functions to the client classes, for example
def persist(objectsofTypeA:Traversable[A],objectsOfTypeB:Traversable[B])
Please also note that according to your application complexity, you might want to expose this logic in different fashions:
as a singleton object in the case your application is simple, and you do not want your persistence logic to be pluggable
through a singleton object which acts as a sort as a "application context", so that in your application at startup you can decide which persistence logic you want to use.
with some sort of lookup service pattern, if your application is distributed.
I know it's not directly possible to serialize a function/anonymous class to the database but what are the alternatives? Do you know any useful approach to this?
To present my situation: I want to award a user "badges" based on his scores. So I have different types of badges that can be easily defined by extending this class:
class BadgeType(id:Long, name:String, detector:Function1[List[UserScore],Boolean])
The detector member is a function that walks the list of scores and return true if the User qualifies for a badge of this type.
The problem is that each time I want to add/edit/modify a badge type I need to edit the source code, recompile the whole thing and re-deploy the server. It would be much more useful if I could persist all BadgeType instances to a database. But how to do that?
The only thing that comes to mind is to have the body of the function as a script (ex: Groovy) that is evaluated at runtime.
Another approach (that does not involve a database) might be to have each badge type into a jar that I can somehow hot-deploy at runtime, which I guess is how a plugin-system might work.
What do you think?
My very brief advice is that if you want this to be truly data-driven, you need to implement a rules DSL and an interpreter. The rules are what get saved to the database, and the interpreter takes a rule instance and evaluates it against some context.
But that's overkill most of the time. You're better off having a little snippet of actual Scala code that implements the rule for each badge, give them unique IDs, then store the IDs in the database.
e.g.:
trait BadgeEval extends Function1[User,Boolean] {
def badgeId: Int
}
object Badge1234 extends BadgeEval {
def badgeId = 1234
def apply(user: User) = {
user.isSufficientlyAwesome // && ...
}
}
You can either have a big whitelist of BadgeEval instances:
val weDontNeedNoStinkingBadges = Map(
1234 -> Badge1234,
5678 -> Badge5678,
// ...
}
def evaluator(id: Int): Option[BadgeEval] = weDontNeedNoStinkingBadges.get(id)
def doesUserGetBadge(user: User, id: Int) = evaluator(id).map(_(user)).getOrElse(false)
... or if you want to keep them decoupled, use reflection:
def badgeEvalClass(id: Int) = Class.forName("com.example.badge.Badge" + id + "$").asInstanceOf[Class[BadgeEval]]
... and if you're interested in runtime pluggability, try the service provider pattern.
You can try and use Scala Continuations - they can give you the ability to serialize the computation and run it at later time or even on another machine.
Some links:
Continuations
What are Scala continuations and why use them?
Swarm - Concurrency with Scala Continuations
Serialization relates to data rather than methods. You cannot serialize functionality because it is a class file which is designed to serialize that and object serialization serializes the fields of an object.
So like Alex says, you need a rule engine.
Try this one if you want something fairly simple, which is string based, so you can serialize the rules as strings in a database or file:
http://blog.maxant.co.uk/pebble/2011/11/12/1321129560000.html
Using a DSL has the same problems unless you interpret or compile the code at runtime.