How to inject quasi quotes array - scala

I have an array of quasi quotes called definitions which I want to inject to the quasi quote tree. How do I go about doing this?
private def generateDaoComponent(files: Array[File]) = {
val file = createNewFile(compDirectory)
val definitions = files.map(f => {
val daoName = f.getName.replace(".java", "")
val daoType = TypeName(daoName)
val daoTerm = TermName(daoName)
q"""def $daoTerm = getValueOrInstantiate($daoName, () => new $daoType(configuration))
"""
})
val tree = q"""
package database.dao {
import org.jooq.SQLDialect
import org.jooq.impl.DefaultConfiguration
import utility.StrongHashMap
trait $componentType extends StrongHashMap[String, Dao] {
this: DaoManager =>
private lazy val configuration = new DefaultConfiguration().set(connection).set(SQLDialect.POSTGRES_9_4)
${definitions.foreach(f => q"${f}")}
}
}"""
writeToFile(file, tree)
}

This is some crazy late night coding, but I found it thanks to this website
3 approaches to Scala code generation
I noticed when it passed the $params array into the quasi quote it used two .. in front of the class constructor like this:
val params = schema.fields.map { field =>
val fieldName = newTermName(field.name)
val fieldType = newTypeName(field.valueType.fullName)
q"val $fieldName: $fieldType"
}
val json = TypeSchema.toJson(schema)
// rewrite the class definition
c.Expr(
q"""
case class $className(..$params) {
def schema = ${json}
}
"""
)
There are two steps to the code I posted in the question to make this working.
1) Convert $definitions.toList to List
2) Add the two .. in front
So the final code looks like this:
val definitions = files.map(f => {
val daoName = f.getName.replace(".java", "")
val daoType = TypeName(daoName)
val daoTerm = TermName(new StringBuilder("get").append(daoName).toString())
q"""def $daoTerm = getValueOrInstantiate($daoName, () => new $daoType(configuration))"""
}).toList <--- HERE!
val tree = q"""
package database.dao {
import org.jooq.SQLDialect
import org.jooq.impl.DefaultConfiguration
import utility.StrongHashMap
trait $componentType extends StrongHashMap[String, Dao] {
this: DaoManager =>
private lazy val configuration = new DefaultConfiguration().set(connection).set(SQLDialect.POSTGRES_9_4)
..$definitions <-- AND HERE!
}
}"""

Related

How to generate Function1 by parameter Class and return Class

I want to register a scala Object Method into spark udf. Now i get the MethodMirror by scala reflect and Parameter by java reflect. But i cannot generate a Function object used to register into spark.udf. Such as:
object ArrayUdfs {
def array2String(arr: Seq[Long])= {
arr.mkString(",")
}
}
I want to register method 'array2String' into spark udf.
First i get the MethodMirror:
def getObjectMethod(clazzPath:String, methodName:String) = {
import scala.reflect.runtime.universe
lazy val runtimeMirror = universe.runtimeMirror(getClass.getClassLoader)
lazy val module = runtimeMirror.staticModule(clazzPath)
lazy val obj = runtimeMirror.reflectModule(module)
lazy val objMirror = runtimeMirror.reflect(obj.instance)
lazy val method = obj.symbol.typeSignature.member(universe.TermName(methodName)).asMethod
lazy val methodObject = objMirror.reflectMethod(method)
methodObject
}
lazy val method = getObjectMethod(clazzPath, funName);
Second. I get Parameter.
def getObjectMethodParams(clazzPath:String, methodName:String): Array[Parameter] = {
val methods = Class.forName(clazzPath).getDeclaredMethods()
var params: Array[Parameter] = null;
methods.foreach(method => {
if (method.getName == methodName) params = method.getParameters
})
params
}
val params = getObjectMethodParams(clazzPath, funName)
Third, register it into spark.udf
val function1: Function1[Seq[String], String] = (arr) => {
method.apply(arr).asInstanceOf[String]
}
spark.udf.register("array2String", function1)
So i want to replace 'Seq[String]' in 'Function1[Seq[String], String]' by 'params' object. The 'String' in 'Function1[Seq[String], String]' is same.

Scala How to Sort List of Objects List by Object field (Object.field)?

I have the following type of data:
case class TipoDeDato[T] (nombreCampo: String,valor: T)
And in my exercise, I need to create the following structure, using the type of data I mentioned:
So, I created the following structure
val registro0: List[TipoDeDato[_>: String with Int]] = List(
new TipoDeDato[String]("Autor", "Gabo"),
new TipoDeDato[String]("Titulo", "100 Años"),
new TipoDeDato[Int]("Numero de Paginas", 700)
)
val registro1: List[TipoDeDato[_>: String with Int]] = List(
new TipoDeDato[String]("Autor", "Gabo"),
new TipoDeDato[String]("Titulo", "Maria"),
new TipoDeDato[Int]("Numero de Paginas", 1200)
)
val registro2: List[TipoDeDato[_>: String with Int]] = List(
new TipoDeDato[String]("Autor", "Gabo"),
new TipoDeDato[String]("Titulo", "Carrasco"),
new TipoDeDato[Int]("Numero de Paginas", 150)
)
val registro3: List[TipoDeDato[_>: String with Int]] = List(
new TipoDeDato[String]("Autor", "Gabo"),
new TipoDeDato[String]("Titulo", "Oceano"),
new TipoDeDato[Int]("Numero de Paginas", 200)
)
And to create the "Libros" object, I have done the following:
val Libros: List[List[TipoDeDato[_>: String with Int]]] = List(registro0,registro1,registro2,registro3)
My question is, how can I sort the "Libros" object, by any of its components, "Autor", "Titulo", "Numero de paginas"?, is this structure adequate for what I need to do?
To sort List of List:
sealed trait TipoDeDato
case class Autor (autor: String) extends TipoDeDato
case class Titulo (titulo: String) extends TipoDeDato
case class NumeroDePaginas (numeroDePaginas: Int) extends TipoDeDato
class TablaItems(var registros: List[List[TipoDeDato]]){
def insertInto(reg: List[List[TipoDeDato]]): TablaItems = {
registros = registros ::: reg
this
}
}
val registro0: List[TipoDeDato] = List(
Autor("HGabo"),
Titulo("ZLa María"),
NumeroDePaginas(752)
)
val registro1: List[TipoDeDato] = List(
Autor("AGabo"),
Titulo("CLa María"),
NumeroDePaginas(521)
)
val Registros1: List[List[TipoDeDato]] = List(registro0)
val Registros2: List[List[TipoDeDato]] = List(registro1)
val tablaLibros = new TablaItems(Registros1)
tablaLibros.registros.foreach(println)
println("----")
tablaLibros.insertInto(Registros2)
tablaLibros.registros.foreach(println)
println("----")
tablaLibros.registros.sortBy(r=>r.collectFirst{
case NumeroDePaginas(n) => n
}.getOrElse(0))
Actually I think you need:
case class Dato(autor: String, titulo: String, numeroDePaginas: Int)
class TablaItems(var registros: List[Dato]){
def insertInto(reg: List[Dato]): TablaItems = {
registros = registros ::: reg
this
}
}
//you can also do (if you prefer) `Dato(author = "HGabo", titulo = "ZLa María", numeroDePaginas = 752)
val registro0 = Dato("HGabo", "ZLa María", 752)
val registro1 = Dato("AGabo", "CLa María", 521)
val Registros1: List[Dato] = List(registro0)
val Registros2: List[Dato] = List(registro1)
val tablaLibros = new TablaItems(Registros1)
tablaLibros.registros.foreach(println)
println("----")
tablaLibros.insertInto(Registros2)
tablaLibros.registros.foreach(println)
println("----")
tablaLibros.registros.sortBy(_.numeroDePaginas)
Also, if this problem requires functional programming (no side-effects, and also I got rid of OOP - however the latter is not mandatory, OOP and FP are orthogonal):
case class TablaItems(registros: List[Dato])
implicit class TablaItemsOperations(tabla: TablaItems){
def withData(reg: List[Dato]) = TablaItems(tabla.registos :: reg)
}
...
val tablaLibros = TablaItems(Registros1)
tablaLibros.registros.foreach(println)
println("----")
val tablaLibrosUpdated = tablaLibros.withData(Registros2)
tablaLibrosUpdated.registros.foreach(println)
println("----")
tablaLibrosUpdated.registros.sortBy(_.numeroDePaginas)

Chisel: Access to Module Parameters from Tester

How does one access the parameters used to construct a Module from inside the Tester that is testing it?
In the test below I am passing the parameters explicitly both to the Module and to the Tester. I would prefer not to have to pass them to the Tester but instead extract them from the module that was also passed in.
Also I am new to scala/chisel so any tips on bad techniques I'm using would be appreciated :).
import Chisel._
import math.pow
class TestA(dataWidth: Int, arrayLength: Int) extends Module {
val dataType = Bits(INPUT, width = dataWidth)
val arrayType = Vec(gen = dataType, n = arrayLength)
val io = new Bundle {
val i_valid = Bool(INPUT)
val i_data = dataType
val i_array = arrayType
val o_valid = Bool(OUTPUT)
val o_data = dataType.flip
val o_array = arrayType.flip
}
io.o_valid := io.i_valid
io.o_data := io.i_data
io.o_array := io.i_array
}
class TestATests(c: TestA, dataWidth: Int, arrayLength: Int) extends Tester(c) {
val maxData = pow(2, dataWidth).toInt
for (t <- 0 until 16) {
val i_valid = rnd.nextInt(2)
val i_data = rnd.nextInt(maxData)
val i_array = List.fill(arrayLength)(rnd.nextInt(maxData))
poke(c.io.i_valid, i_valid)
poke(c.io.i_data, i_data)
(c.io.i_array, i_array).zipped foreach {
(element,value) => poke(element, value)
}
expect(c.io.o_valid, i_valid)
expect(c.io.o_data, i_data)
(c.io.o_array, i_array).zipped foreach {
(element,value) => poke(element, value)
}
step(1)
}
}
object TestAObject {
def main(args: Array[String]): Unit = {
val tutArgs = args.slice(0, args.length)
val dataWidth = 5
val arrayLength = 6
chiselMainTest(tutArgs, () => Module(
new TestA(dataWidth=dataWidth, arrayLength=arrayLength))){
c => new TestATests(c, dataWidth=dataWidth, arrayLength=arrayLength)
}
}
}
If you make the arguments dataWidth and arrayLength members of TestA you can just reference them. In Scala this can be accomplished by inserting val into the argument list:
class TestA(val dataWidth: Int, val arrayLength: Int) extends Module ...
Then you can reference them from the test as members with c.dataWidth or c.arrayLength

Slick code generation for only a single schema

Is there a way to have Slick's code generation generate code for only a single schema? Say, public? I have extensions that create a whole ton of tables (eg postgis, pg_jobman) that make the code that slick generates gigantic.
Use this code with appropriate values and schema name,
object CodeGenerator {
def outputDir :String =""
def pkg:String =""
def schemaList:String = "schema1, schema2"
def url:String = "dburl"
def fileName:String =""
val user = "dbUsername"
val password = "dbPassword"
val slickDriver="scala.slick.driver.PostgresDriver"
val JdbcDriver = "org.postgresql.Driver"
val container = "Tables"
def generate() = {
val driver: JdbcProfile = buildJdbcProfile
val schemas = createSchemaList
var model = createModel(driver,schemas)
val codegen = new SourceCodeGenerator(model){
// customize Scala table name (table class, table values, ...)
override def tableName = dbTableName => dbTableName match {
case _ => dbTableName+"Table"
}
override def code = {
//imports is copied right out of
//scala.slick.model.codegen.AbstractSourceCodeGenerator
val imports = {
"import scala.slick.model.ForeignKeyAction\n" +
(if (tables.exists(_.hlistEnabled)) {
"import scala.slick.collection.heterogenous._\n" +
"import scala.slick.collection.heterogenous.syntax._\n"
} else ""
) +
(if (tables.exists(_.PlainSqlMapper.enabled)) {
"import scala.slick.jdbc.{GetResult => GR}\n" +
"// NOTE: GetResult mappers for plain SQL are only generated for tables where Slick knows how to map the types of all columns.\n"
} else ""
) + "\n\n" //+ tables.map(t => s"implicit val ${t.model.name.table}Format = Json.format[${t.model.name.table}]").mkString("\n")+"\n\n"
}
val bySchema = tables.groupBy(t => {
t.model.name.schema
})
val schemaFor = (schema: Option[String]) => {
bySchema(schema).sortBy(_.model.name.table).map(
_.code.mkString("\n")
).mkString("\n\n")
}
}
val joins = tables.flatMap( _.foreignKeys.map{ foreignKey =>
import foreignKey._
val fkt = referencingTable.TableClass.name
val pkt = referencedTable.TableClass.name
val columns = referencingColumns.map(_.name) zip
referencedColumns.map(_.name)
s"implicit def autojoin${fkt + name.toString} = (left:${fkt} ,right:${pkt}) => " +
columns.map{
case (lcol,rcol) =>
"left."+lcol + " === " + "right."+rcol
}.mkString(" && ")
})
override def entityName = dbTableName => dbTableName match {
case _ => dbTableName
}
override def Table = new Table(_) {
table =>
// customize table value (TableQuery) name (uses tableName as a basis)
override def TableValue = new TableValue {
override def rawName = super.rawName.uncapitalize
}
// override generator responsible for columns
override def Column = new Column(_){
// customize Scala column names
override def rawName = (table.model.name.table,this.model.name) match {
case _ => super.rawName
}
}
}
}
println(outputDir+"\\"+fileName)
(new File(outputDir)).mkdirs()
val fw = new FileWriter(outputDir+File.separator+fileName)
fw.write(codegen.packageCode(slickDriver, pkg, container))
fw.close()
}
def createModel(driver: JdbcProfile, schemas:Set[Option[String]]): Model = {
driver.simple.Database
.forURL(url, user = user, password = password, driver = JdbcDriver)
.withSession { implicit session =>
val filteredTables = driver.defaultTables.filter(
(t: MTable) => schemas.contains(t.name.schema)
)
PostgresDriver.createModel(Some(filteredTables))
}
}
def createSchemaList: Set[Option[String]] = {
schemaList.split(",").map({
case "" => None
case (name: String) => Some(name)
}).toSet
}
def buildJdbcProfile: JdbcProfile = {
val module = currentMirror.staticModule(slickDriver)
val reflectedModule = currentMirror.reflectModule(module)
val driver = reflectedModule.instance.asInstanceOf[JdbcProfile]
driver
}
}
I encountered the same problem and I found this question. The answer by S.Karthik sent me in the right direction. However, the code in the answer is slightly outdated. And I think a bit over-complicated. So I crafted my own solution:
import slick.codegen.SourceCodeGenerator
import slick.driver.JdbcProfile
import slick.model.Model
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, ExecutionContext}
val slickDriver = "slick.driver.PostgresDriver"
val jdbcDriver = "org.postgresql.Driver"
val url = "jdbc:postgresql://localhost:5432/mydb"
val outputFolder = "/path/to/src/test/scala"
val pkg = "com.mycompany"
val user = "user"
val password = "password"
object MySourceCodeGenerator {
def run(slickDriver: String, jdbcDriver: String, url: String, outputDir: String,
pkg: String, user: Option[String], password: Option[String]): Unit = {
val driver: JdbcProfile =
Class.forName(slickDriver + "$").getField("MODULE$").get(null).asInstanceOf[JdbcProfile]
val dbFactory = driver.api.Database
val db = dbFactory.forURL(url, driver = jdbcDriver, user = user.orNull,
password = password.orNull, keepAliveConnection = true)
try {
// **1**
val allSchemas = Await.result(db.run(
driver.createModel(None, ignoreInvalidDefaults = false)(ExecutionContext.global).withPinnedSession), Duration.Inf)
// **2**
val publicSchema = new Model(allSchemas.tables.filter(_.name.schema.isEmpty), allSchemas.options)
// **3**
new SourceCodeGenerator(publicSchema).writeToFile(slickDriver, outputDir, pkg)
} finally db.close
}
}
MySourceCodeGenerator.run(slickDriver, jdbcDriver, url, outputFolder, pkg, Some(user), Some(password))
I'll explain what's going on here:
I copied the run function from the SourceCodeGenerator class that's in the slick-codegen library. (I used version slick-codegen_2.10-3.1.1.)
// **1**: In the origninal code, the generated Model was referenced in a val called m. I renamed that to allSchemas.
// **2**: I created a new Model (publicSchema), using the options from the original model, and using a filtered version of the tables set from the original model. It turns out tables from the public schema don't get a schema name in the model. Hence the isEmpty. Should you need tables from one or more other schemas, you can easily create a different filter expression.
// **3**: I create a SourceCodeGenerator with the created publicSchema model.
Of course, it would even be better if the Slick codegenerator could incorporate an option to select one or more schemas.

Scala/Play: load template dynamically

I have this Scala/Play application and I have to fetch a bunch of templates via AJAX. I'm doing something like this now:
def home = Action {
Ok(views.html.home())
}
def about = Action {
Ok(views.html.about())
}
def contact = Action {
Ok(views.html.contact())
}
//etc
But this is just creating an action for every template. Can I do something like this instead:
def loadTemplate(templateName) = Action {
//Load template from "views" with name being value of parameter templateName
}
Is this possible on Play Framework? If so then how?
Play Framework 2.2.1 / Scala 2.10.3 / Java 8 64bit
UPDATE: My original question might have been misunderstood. I don't want to compile a template, I want to fetch already compiled one in a more dynamic way.
UPDATE2: I think I found something very close, if not exactly what I need on this answer, but it's in Java and I need it in Scala.
Using scala reflection:
object Application extends Controller {
import reflect.runtime.universe._
val currentMirror = runtimeMirror(Play.current.classloader)
val packageName = "views.html."
def index(name: String) = Action {
val templateName = packageName + name
val moduleMirror = currentMirror.reflectModule(currentMirror.staticModule(templateName))
val methodSymbol = moduleMirror.symbol.typeSignature.declaration(newTermName("apply")).asMethod
val instanceMirror = currentMirror.reflect(moduleMirror.instance)
val methodMirror = instanceMirror.reflectMethod(methodSymbol)
Ok(methodMirror.apply().asInstanceOf[Html])
}
}
Based on #Nilanjan answer and for play-2.4 scala-2.11.7
def page(page: String) = Action.async { implicit request =>
Future.successful(Ok(loadTemplate(page)))
}
import reflect.runtime.universe._
import play.api._
import play.twirl.api.Html
val currentMirror = runtimeMirror(Play.current.classloader)
val packageName = "views.html."
private def loadTemplate(name: String) = {
val templateName = packageName + name
val moduleMirror = currentMirror.reflectModule(currentMirror.staticModule(templateName))
val methodSymbol = moduleMirror.symbol.info.member(TermName("apply")).asMethod
val instanceMirror = currentMirror.reflect(moduleMirror.instance)
val methodMirror = instanceMirror.reflectMethod(methodSymbol)
methodMirror.apply().asInstanceOf[Html]
}
Here we go again! #Nilanjan answer adopted to Scala 2.12 and Play 2.6 (as moving to DI play.api.Play.current, TermName and declaration have been deprecated). For app, I had to use injector because #Inject()(app: Application) was causing circular dependency
class HomeController #Inject()(injector: Injector, cc: ControllerComponents) extends AbstractController(cc) {
def index(name: String = "index") = Action { implicit request: Request[AnyContent] =>
import reflect.runtime.universe._
val app = injector.instanceOf[Application]
val currentMirror = runtimeMirror(app.classloader)
val packageName = "views.html."
val templateName = packageName + name
val moduleMirror = currentMirror.reflectModule(currentMirror.staticModule(templateName))
val methodSymbol = moduleMirror.symbol.typeSignature.decl(TermName("apply")).asMethod
val instanceMirror = currentMirror.reflect(moduleMirror.instance)
val methodMirror = instanceMirror.reflectMethod(methodSymbol)
Ok(methodMirror.apply("some", "content").asInstanceOf[Html])
}
}
It's not quite good idea to allow search view by any text (for security reasons) as such can be passed in param, instead, you can resolve this quite easy with match statement - it will allow you restrict request to allowed views only and will handle wrong requests as well, probably Scala geeks can demonstrate nicer code, but this will work for you out of the box:
def loadTemplate(templateName: String) = Action {
templateName match {
case "about" => Ok(about())
case "home" => Ok(home())
case "contact" => Ok(contact())
case _ => NotFound(notFoundView())
}
}
route:
GET /load-template/:templateName controllers.Application.loadTemplate(templateName)
Additional benefit is that you can grab additional data from request and pass it to the resolved view depending on templateName param
In your Template file
general_template.scala.html
#(templateName : String = "none")
#templateName match {
case "about" => { //html elements}
case "home" => {//html elements}
case "contact" => {//html elements}
case _ => { }
}
and in your controller
def loadTemplate(templateName) = Action {
Ok(views.html.general_template(templateName))
}