How to implement Generic slick dao with oauth2 in scala - scala

To work with OAuth2, Slick, Akka-HTTP in scala, I'm using the code from cdiniz https://github.com/cdiniz/slick-akka-http-oauth2.
I tried to create a DAO: UserDao and respective API: UserApi to fetch all records(in JSON) from database after successful verification of OAuth2.
BaseDao Code:
trait BaseDao[T,A] {
...
def getAll(): Future[Seq[A]]
}
class BaseDaoImpl[T <: BaseTable[A], A <: BaseEntity]()(implicit val tableQ: TableQuery[T], implicit val db: JdbcProfile#Backend#Database,implicit val profile: JdbcProfile) extends BaseDao[T,A] with Profile with DbModule {
import profile.api._
...
override def getAll(): Future[Seq[A]] = {
db.run(tableQ.result)
}
}
UserDao Code:
object UserService {
val modules = new ConfigurationServiceImpl with ActorServiceImpl with PersistenceServiceImpl
import modules._
class UserDao extends BaseDaoImpl[UserTable,UserEntity1] {}
}
UserApi Code:
//using io.circe for json encoder
class UserApi(val modules: Configuration with PersistenceService, val db: DatabaseService)(implicit executionContext: ExecutionContext) extends BaseApi {
override val oauth2DataHandler = modules.oauth2DataHandler
val userService = new UserDao
val testApi = pathPrefix("auth") {
path("users") {
pathEndOrSingleSlash {
get {
authenticateOAuth2Async[AuthInfo[OauthAccount]]("realm", oauth2Authenticator) {
auth => complete(userService.getAll().map(_.asJson))
}
}
}
}
}
}
Supporting Codes:
trait Profile {
val profile: JdbcProfile
}
trait DbModule extends Profile{
val db: JdbcProfile#Backend#Database
}
trait PersistenceService {
val accountsDao: AccountsDao
val oAuthAuthorizationCodesDao: OAuthAuthorizationCodesDao
val oauthClientsDao: OAuthClientsDao
val oauthAccessTokensDao: OAuthAccessTokensDao
val oauth2DataHandler : DataHandler[OauthAccount]
}
trait PersistenceServiceImpl extends PersistenceService with DbModule{
this: Configuration =>
private val dbConfig : DatabaseConfig[JdbcProfile] = DatabaseConfig.forConfig[JdbcProfile]("mysqldb")
override implicit val profile: JdbcProfile = dbConfig.driver
override implicit val db: JdbcProfile#Backend#Database = dbConfig.db
override val accountsDao = new AccountsDaoImpl
override val oAuthAuthorizationCodesDao = new OAuthAuthorizationCodesDaoImpl
override val oauthClientsDao = new OAuthClientsDaoImpl(this)
override val oauthAccessTokensDao = new OAuthAccessTokensDaoImpl(this)
override val oauth2DataHandler = new OAuth2DataHandler(this)
}
Here, The code runs without any issue but doesn't get any records as the response though there are records.
What is the problem in this code that I cannot look into? Any suggestion would be appreciable.

Related

slick 3.2 - how to define an "abstract" trait with tables and queries definitions but no concrete profile?

I want to define the queries and tables without the direct import of the final database profile (H2, MySQL etc) - so in unit tests I would use H2, and for the staging/production I would use MySQL. So far I couldn't find a way to import all necessary abstract components to get this working:
import slick.jdbc.H2Profile.api._
class OAuthCredentialsTable(tag: Tag) extends Table[OAuth](tag, "credentials_oauth") {
def username: Rep[String] = column[String]("username", O.SqlType("VARCHAR"))
def service: Rep[String] = column[String]("service", O.SqlType("VARCHAR"))
def serviceId: Rep[String] = column[String]("service_id", O.SqlType("VARCHAR"))
def userRef: ForeignKeyQuery[UserTable, User] = foreignKey("oauth_user_fk", username, userTable)(_.username, onDelete = ForeignKeyAction.Cascade)
override def * = (username, service, serviceId) <> (OAuth.tupled, OAuth.unapply)
}
val oauthTable: TableQuery[OAuthCredentialsTable] = TableQuery[OAuthCredentialsTable]
Eventually I discovered that to accomplish the driver-agnostic setup, it could be done as simple as that:
object UserPersistence extends JdbcProfile {
import api._
class UserTable(tag: Tag) extends Table[User](tag, "users") {
def username: Rep[String] = column[String]("username", O.PrimaryKey, O.SqlType("VARCHAR"))
def password: Rep[String] = column[String]("password", O.SqlType("VARCHAR"))
def serverkey: Rep[String] = column[String]("serverkey", O.SqlType("VARCHAR"), O.Length(64))
def salt: Rep[String] = column[String]("salt", O.SqlType("VARCHAR"), O.Length(64))
def iterations: Rep[Int] = column[Int]("iterationcount", O.SqlType("INT"))
def created: Rep[Timestamp] = column[Timestamp]("created_at", O.SqlType("TIMESTAMP"))
val mkUser: ((String, String, String, String, Int, Timestamp)) ⇒ User = {
case ((name, pwd, _, _, _, created)) ⇒ User(name, pwd, created.toInstant)
}
def unMkUser(u: User) = Some(u.username, u.password, "", "", 0, new Timestamp(u.createdAt.toEpochMilli))
override def * = (username, password, serverkey, salt, iterations, created) <> (mkUser, unMkUser)
}
val userTable: TableQuery[UserTable] = TableQuery[UserTable]
}
and then in order to use different profiles - you need to supply different database implementation when you run something, e.g
trait UserPersistence {
protected def db: Database
protected implicit val ec: ExecutionContext
override def findCredentialsOrRegister(oauthCredentials: OAuth): Future[User] = {
val userFound = (for (
creds ← oauthTable.filter(x ⇒ x.service === oauthCredentials.service && x.serviceId === oauthCredentials.serviceId);
user ← userTable.filter(x ⇒ x.username === creds.username)
) yield user).result
val authenticate = userFound.flatMap {
case Seq(user) ⇒
DBIO.from(Future.successful(user))
case Seq() ⇒
val newUUID = UUID.randomUUID
val user = User(newUUID.toString, UUID.randomUUID().toString, Instant.now())
DBIO.seq(
userTable += user,
oauthTable += oauthCredentials.copy(username = newUUID.toString)
) andThen DBIO.from(Future.successful(user))
}
db.run(authenticate.transactionally)
}
and then in test
val impl = new UserPersistence {
override def db: H2Profile.api.Database = // initialize and populate the database
override implicit val ec: ExecutionContext = scala.concurrent.ExecutionContext.global
}
For MySql just assign the MySQL profile to db property (and change type).
I hope I got you right, you can achieve it with injections and your configuration, separating test config from prod,
So I think you can do something like this-
Create binding for injection-
class DbModule extends Module {
bind[slick.driver.JdbcProfile#Backend#Database] toNonLazy DatabaseConfigProvider.get[JdbcProfile](inject[Application]).db
}
Then (for example)-
abstract class BaseDaoImpl[T <: IdentifiableTable[S] : ClassTag, S <: RichEntity[S]](implicit injector: Injector)
extends BaseDao[S] with Injectable with Logger{
protected val db: slick.driver.JdbcProfile#Backend#Database = inject[slick.driver.JdbcProfile#Backend#Database]
protected val table: TableQuery[T]
def insert(obj: S): Future[S] = {
val insertQuery = table.returning(table.map(_.id)) into ((item, id) => item.withDifferentId(Some(id)))
val res = db.run(insertQuery += obj)
logger.debug(s"Inserting ${this.getClass} - ${res.toString}")
res
}
def all(): Future[Seq[S]] = {
db.run(table.result)
}
}
I used this configuration in order to change the DB profile in a application.conf file.
import slick.jdbc.JdbcProfile
import slick.basic._
trait DBComponent {
// use the application.conf to change the profile
val database = DatabaseConfig.forConfig[JdbcProfile]("h2")
val driver = database.profile
}
trait BankTable extends DBComponent {
import driver.api._
class BankTable(tag: Tag) extends Table[Bank](tag, "bank") {
val id = column[Int]("id", O.PrimaryKey, O.AutoInc)
val name = column[String]("name")
def * = (name, id.?) <> (Bank.tupled, Bank.unapply)
}
protected val bankTableQuery = TableQuery[BankTable]
protected def bankTableAutoInc = bankTableQuery returning
bankTableQuery.map(_.id)
}
class BankRepositoryBDImpl extends BankTable with BankRepository {
import driver.api._
val db = database.db
def createBank(bank: Bank): Future[Int] = db.run { bankTableAutoInc += bank }
}
and use a application.conf file
h2 = {
url = "jdbc:h2:mem:test1"
driver = org.h2.Driver
connectionPool = disabled
keepAliveConnection = true
}
sqlite = {
url = "jdbc:sqlite::memory:"
driver = org.sqlite.JDBC
connectionPool = disabled
keepAliveConnection = true
}
To accomplish this in my project I created an object out of the JdbcProfile trait and imported that everywhere:
JdbcProfile.scala:
package my.application.data.support
import slick.jdbc.JdbcProfile
object JdbcProfile extends JdbcProfile
Wherever I define tables I import it as follows:
import my.application.data.support.JdbcProfile.api._
...
In order to support extensions to slick, I created an abstraction called DatabaseSupport where each database type would have their own concrete implementation which is injected at startup-time depending on configuration.
DatabaseSupport.scala:
package my.application.data.support
/**
* Database support implicits and methods.
*/
trait DatabaseSupport {
/**
* Implicit database array type.
*/
implicit val seqStringType: JdbcProfile.DriverJdbcType[Seq[String]]
}
This way I can have seperate H2DatabaseSupport.scala and PostgresDatabaseSupport.scala implementations that specifies seqStringType in database-specific ways.

Slick database configuration

I have an issue with Slick configuration in my Play application (Play 2.4.3).
I read documentation article but want to move dbConfig from controller to specified trait and mixin this trait to repository class.
There are several files in project: ClientRepository (class), BaseClientRepository (trait) and BaseDbRepository (trait).
trait BaseDbRepository {
val dbConfig = DatabaseConfigProvider.get[JdbcProfile](Play.current)
import dbConfig.driver.api._
def withConnection[T](f: => DBIOAction[T, NoStream, Nothing]) = {
dbConfig.db.run(f)
}
}
trait BaseClientRepository {
def getById(id: Int): Future[Client]
def getByLocation(location: String): Future[Seq[Client]]
}
class ClientRepository extends BaseDbRepository with BaseClientRepository {
def getById: Future[Client] = withConnection {
...
}
def getByLocation: Future[Seq[Client]] = withConnection {
...
}
}
This works great with my Client controller:
class Client extends Controller {
def getById(id: Int) = ???
}
But when I try to use DI with Guice:
class Client #Inject()(clientRepository: BaseClientRepository) extends Controller {
def getById(id: Int) = Action.async {
// I try to use client repository here
}
}
It failes with the following exception
CreationException: Unable to create injector, see the following errors:
1) An exception was caught and reported. Message: There is no started application
at com.google.inject.util.Modules$OverrideModule.configure(Modules.java:177)
I tried to move this definition val dbConfig = DatabaseConfigProvider.get[JdbcProfile](Play.current) into Global.scala and it just works, but as I now Global.scala is deprecated now.
So, where is the best place for it?
Update: I use injection module for DI configuration:
class InjectionModule extends AbstractModule {
def configure() {
bind(classOf[BaseClientRepository]).toInstance(new ClientRepository)
}
}
dbConfig should be a lazy val or a function in this case.
That works for me:
private lazy val dbConfig = DatabaseConfigProvider.get[JdbcProfile](Play.current)
Guice failed to inject an implementation of BaseClientRepository, the annotation #ImplementedBy can help.
#ImplementedBy(classOf[ClientRepository])
trait BaseClientRepository {
def getById(id: Int): Future[Client]
def getByLocation(location: String): Future[Seq[Client]]
}

Scala: class type required but {generic type} found in trait

I'm using Slick 3.0 and trying to create a trait to offer basic operations. Here is my trait:
object DAO {
var db: Database = null
}
trait CommonAPI[T <: Table[_]] {
private val db = DAO.db
private val objects = TableQuery[T]
def count: Future[Int] = db.run(objects.length.result)
def insert(obj: T#TableElementType): Future[Int] = db.run(objects += obj)
def all: Future[Seq[T]] = db.run(objects.result)
}
DAO.db is initialized in Play's onStart method. However, I met compilation error class type required but T found in line private val objects = TableQuery[T].
What am I supposed to do? Thanks!
Here is one solution:
At first, define this to avoid class type issue..
class Service[T <: Table[_]](path: String, cons: Tag => T){
lazy val db = Database.forConfig(path)
def query = TableQuery[T](cons)
}
Then use it this way, Post is sub class of Table:
object Abcd {
object Def extends Service[Post]("mydb", abc) {
def test = {
//db
val q = query.drop(1).take(20)
val r = db.run(q.result)
println(q.result.statements.head)
println(r)
r
}
}
private def abc(tag: Tag) = new Post(tag)
}
This solution tested ok.
Hi I have done something similar. I used a bit different approach. I have one generic DAO and few(per resource/table) classes which just inherits for it.
this is generic DAO:
class BaseDbEntity[T <: BaseEntity, R <: BaseT[T]](val tableName: String, tableQuery: TableQuery[R]) extends DatabaseAccess{
val createReturningId = tableQuery returning tableQuery.map{item => item.id}
def create(entity: T): Int = {
connectionPool withSession {
implicit session =>
val resId = createReturningId += entity
resId
}
}
def getAll = {
connectionPool withSession {
implicit session =>
tableQuery.list
}
}
}
code of full class:
(https://github.com/vixxx123/scalasprayslickexample/blob/master/src/main/scala/com/vixxx123/scalasprayslickexample/database/BaseDbEntity.scala)
and Specific DAO with corresponding Table class:
class PersonDao extends BaseDbEntity[Person, PersonT]("person", TableQuery[PersonT])
class PersonT(tag: Tag) extends BaseT[Person](tag, "person") {
def name: Column[String] = column[String]("name")
def lastname: Column[String] = column[String]("lastname")
override def * = (id.?, name, lastname) <> (
(Person.apply _).tupled, Person.unapply)
}
you can find that class here: https://github.com/vixxx123/scalasprayslickexample/blob/master/src/main/scala/com/vixxx123/scalasprayslickexample/exampleapi/person/PersonDao.scala
Maybe it will help you out.
You can pass a tag instead and create table query from it , if your table class is defined as :
case class Sample(....)
class SampleTable(tag: Tag)
extends Table[Sample](tag, "sample_table") {
.....
}
then you can implement your generic trait as below :
import scala.slick.driver.MySQLDriver.simple.Tag // here mysql is used , you can import the driver specific to your db
object DAO {
var db: Database = null
}
trait CommonAPI[T, A<: Table[T]] {
private val db = DAO.db
private val tableTag : Tag => A = _
def setTag(tag : Tag => A) = { tableTag = tag }
private val objects = TableQuery(tableTag)
def count: Future[Int] = db.run(objects.length.result)
def insert(obj: T#TableElementType): Future[Int] = db.run(objects += obj)
def all: Future[Seq[T]] = db.run(objects.result)
}

How do I use infinite Scala streams as source in Spark Streaming?

Suppose I essentially want Stream.from(0) as InputDStream. How would I go about this? The only way I can see is to use StreamingContext#queueStream, but I'd have to either enqueue elements from another thread or subclass Queue to create a queue that behaves like an infinite stream, both of which feel like a hack.
What's the correct way to do this?
I don't think that it's available in Spark by default but it's easy to implement it with ReceiverInputDStream.
import org.apache.spark.storage.StorageLevel
import org.apache.spark.streaming.StreamingContext
import org.apache.spark.streaming.dstream.ReceiverInputDStream
import org.apache.spark.streaming.receiver.Receiver
class InfiniteStreamInputDStream[T](
#transient ssc_ : StreamingContext,
stream: Stream[T],
storageLevel: StorageLevel
) extends ReceiverInputDStream[T](ssc_) {
override def getReceiver(): Receiver[T] = {
new InfiniteStreamReceiver(stream, storageLevel)
}
}
class InfiniteStreamReceiver[T](stream: Stream[T], storageLevel: StorageLevel) extends Receiver[T](storageLevel) {
// Stateful iterator
private val streamIterator = stream.iterator
private class ReadAndStore extends Runnable {
def run(): Unit = {
while (streamIterator.hasNext) {
val next = streamIterator.next()
store(next)
}
}
}
override def onStart(): Unit = {
new Thread(new ReadAndStore).run()
}
override def onStop(): Unit = { }
}
Slightly modified code tat works with Spark 2.0:
import org.apache.spark.storage.StorageLevel
import org.apache.spark.streaming.StreamingContext
import org.apache.spark.streaming.dstream.ReceiverInputDStream
import org.apache.spark.streaming.receiver.Receiver
import scala.reflect.ClassTag
class InfiniteDStream[T: ClassTag](
#transient ssc_ : StreamingContext,
stream: Stream[T],
storageLevel: StorageLevel
) extends ReceiverInputDStream[T](ssc_) {
override def getReceiver(): Receiver[T] = {
new InfiniteStreamReceiver(stream, storageLevel)
}
}
class InfiniteStreamReceiver[T](stream: Stream[T], storageLevel: StorageLevel) extends Receiver[T](storageLevel) {
private class ReadAndStore extends Runnable {
def run(): Unit = {
stream.foreach(store)
}
}
override def onStart(): Unit = {
val t = new Thread(new ReadAndStore)
t.setDaemon(true)
t.run()
}
override def onStop(): Unit = {}
}

How do I improve this attempt at a cake pattern

This is one of my early attempts at implementing a Scala Cake Pattern:
trait dbConfig {
val m: Model = ???
}
trait testDB extends dbConfig {
override val m = new Model(Database.forURL("jdbc:h2:mem:testdb", driver = "org.h2.Driver"))
m.cleanDB
}
trait productionDB extends dbConfig {
override val m = new Model(Database.forURL("jdbc:postgresql:silly:productionDB", driver = "org.postgresql.Driver"))
}
trait SillySystem extends HttpService with dbConfig {
....
// System logic
....
}
This will allow me to use my service like this while testing:
class TestService extends SillySystem with testDB {
.....
}
And like this for production:
class ProductionService extends SillySystem with productionDB {
.....
}
This works, but am I doing it correctly?
It could be helpful to make DbConfig abstract and use def since one can override a def with a val or lazy val, but not the other way round.
SillySystem is not a DbConfig, so use dependency injection instead of inheritance.
trait DbConfig {
def m: Model // abstract
}
trait TestDB extends DbConfig {
// you can override def with val
val m = new Model(Database.forURL("jdbc:h2:mem:testdb", driver = "org.h2.Driver"))
m.cleanDB
}
trait ProductionDB extends DbConfig {
val m = new Model(Database.forURL("jdbc:postgresql:silly:productionDB", driver = "org.postgresql.Driver"))
}
trait SillySystem extends HttpService {
this: DbConfig => // self-type. SillySystem is not a DbConfig, but it can use methods of DbConfig.
....
// System logic
....
}
val testService = new SillySystem with TestDB
val productionService = new SillySystem with ProductionDB
val wrongService1 = new SillySystem // error: trait SillySystem is abstract; cannot be instantiated
val wrongService2 = new SillySystem with DbConfig // error: object creation impossible, since method m in trait DbConfig of type => Model is not defined
val correctService = new SillySystem with DbConfig { val m = new Model(...) } // correct