I'm developing a library and want to showcase some code in a small website. The code-snippets should be sourced directly from the same project, i.e. the code needs to be self-referencing.
I want to write a macro that, given a file or a classname returns a string with the file contents. How can I achieve this?
I would also accept giving a classname or type as argument
Are you looking for such a macro?
import scala.quoted.*
inline def getContent[A]: String = ${getContentImpl[A]}
def getContentImpl[A: Type](using Quotes): Expr[String] =
import quotes.reflect.*
// for the source specifically of A
// val str = TypeRepr.of[A].typeSymbol.tree.pos.sourceCode.getOrElse(
// report.errorAndAbort("no source code")
// )
// for the whole source file containing the definition of A
val str = TypeRepr.of[A].typeSymbol.pos.getOrElse(
report.errorAndAbort("no symbol position")
).sourceFile.content.getOrElse(
report.errorAndAbort("no source-file content")
)
Expr(str)
// in a different file
getContent[A]
//import Macros.getContent
//
//object Main extends App {
// class A:
// def foo = 1
//
// println(
// getContent[A]
// )
//}
Related
I want to auto-generate an overriden method in a compiler plugin (Scala 3) for a class like:
trait SpecialSerialize {
def toJson(sb: StringBuilder, c:SJConfig): Unit = {println("wrong")}
}
case class Person(name:String, age:Int) extends SpecialSerialize
The plugin would generate:
case class Person(name:String, age:Int) extends SpecialSerialize {
override def toJson(sb: StringBuilder, c:SJConfig): Unit = ... // code here
}
I have a phase:
class ReflectionWorkerPhase extends PluginPhase {
import tpd._
val phaseName = "reflectionWorker"
override val runsAfter = Set(Pickler.name)
override def transformTypeDef(tree: TypeDef)(implicit ctx: Context): Tree =
if tree.isClassDef && !tree.rhs.symbol.isStatic then // only look at classes
// 0. Get a FreshContext so we can set the tree to this tree. (for '{} later)
implicit val fresh = ctx.fresh
fresh.setTree(tree)
QuotesCache.init(fresh)
implicit val quotes:Quotes = QuotesImpl.apply() // picks up fresh
import quotes.reflect.*
// 1. Set up method symbol, define parameters and return type
val toJsonSymbol = Symbol.newMethod(
Symbol.spliceOwner,
"toJson",
MethodType(
List("sb","config"))( // parameter list
_ => List( // types of the parameters
TypeRepr.of[StringBuilder],
TypeRepr.of[SJConfig],
),
_ => TypeRepr.typeConstructorOf(classOf[Unit]) // return type
),
Flags.Override, // Note override here
Symbol.noSymbol
)
// 2. Get our class' Symbol for ownership reassignment
val classDef = tree.asInstanceOf[ClassDef]
val classSymbol = classDef.symbol
// 3. Define our method definition (DefDef) using our method symbol defined above
val toJsonMethodDef = DefDef(
toJsonSymbol,
{
case List(List(sb: Term, config: Term)) =>
given Quotes = toJsonSymbol.asQuotes
Some({
// Multiple quotes here intentional...
// Real code will generate a list of quoted statements
quoted.Expr.ofList(List(
'{ println("Hello") },
'{ println("World") }
))
}.asTerm.changeOwner(toJsonSymbol))
}
).changeOwner(classSymbol)
// 4. Add toJsonMethodDef to tree and return
val cd = ClassDef.copy(classDef)(
name = classDef.name,
constr = classDef.constructor,
parents = classDef.parents,
selfOpt = classDef.self,
body = toJsonMethodDef +: classDef.body
)
cd.asInstanceOf[dotty.tools.dotc.ast.tpd.Tree]
else
tree
}
When I use the plugin on a sample Person class, I get this error on compile:
Exception in thread "sbt-bg-threads-1" java.lang.ClassFormatError: Duplicate method name "toJson" with signature "(Lscala.collection.mutable.StringBuilder;Lco.blocke.scala_reflection.SJConfig;)V" in class file com/foo/Person
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1013)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
...
So something in the other compile phases failed to recognize my generated method as being an override and tried to copy in the "master" method from the trait, and of course the JVM lost its mind at runtime, finding 2 copies of toJson with the same signature.
How can I fix this so my generated method is recognized as a valid override such that a 2nd toJson method won't be generated in a later phase?
quick question on calling an object and running it from main just need the last command val op = df.coalesce(1).write.mode("overwrite").format("csv").save("report")to be able called from a main class
how do we intilize an instance of object and run to get to write to csv?
def run(implicit context: Context): Unit = {
val timer = Timer.start()
//not working
val newRep = Report_Adhoc
val d = newRep.tab.toDF()
val op = d.coalesce(1).write.mode("overwrite").format("csv").save("report")
println(s"pipeline complete in [${timer.elapsedTime()}]")
}
Main class is this ^ but this throws null point exception
object Report_Adhoc extends App with TransientLogger{
// code not including too verbose
val tab =
counts
.filter(c => c._1.id.nonEmpty && c._2.id.nonEmpty)
.map(c => (c._1, c._2, c._3, c._3.values.sum))
.sort($"_4".desc)
.map(count =>
row(
count._1.id, count._1.label,
count._2.id, count._2.label,
count._3(CITE), count._3(CROSS), count._3(MANUAL),
count._3(RECIPROCAL), count._3(TRANSITIVE), count._3(FAMILY),
count._4
)
)
val df = tab.toDF()
val op = df.coalesce(1).write.mode("overwrite").format("csv").save("report")
}
take a look at this:
https://www.scala-lang.org/api/2.12.3/scala/DelayedInit.html
https://www.scala-lang.org/api/2.12.3/scala/App.html
Also, don't put underscore in names. don't reuse main class/object. Last but not least, make a minimum reproduction instead of pasting big block of code like this.
Need to extract tokens and fixed text. Example:
"Hello {token1} today's date is {token2} would you like to call {token3}"
would return
FixedPart("Hello ")
TokenPart(token1)
FixedPart(" today's date is ")
TokenPart(token2)
FixedPart(" would you like to call ")
TokenPart(token3)
Here is naive implementation
import org.parboiled2.ParserInput
import org.parboiled2.Parser
import org.parboiled2.CharPredicate
sealed trait Part
case class TokenPart(tokenName : String ) extends Part
case class FixedPart( text : String ) extends Part
class MyParser(val input: ParserInput) extends Parser {
def Token = rule { '{' ~ capture(TokenName) ~> (TokenPart(_)) ~'}' }
//how this should be implemented??
def NotToken = rule { capture (!Token) ~>(FixedPart(_) )}
def TokenName = rule { CharPredicate.Alpha ~ oneOrMore (CharPredicate.AlphaNum) }
// This would not work
def TokenNotToken = rule { (Token|NotToken) }
def InputLine = rule { zeroOrMore (TokenNotToken) }
}
object MyParser {
def main(args: Array[String]) {
val res = new MyParser("Hello {token1} today's date is {token2} would you like to call {token3}").InputLine.run() // Success
println( res )
}
}
Any other to implement this ??
Hi I modified your code and added some comments (I hope they will be helpful), so it works, and (I guess) does what you wanted it to do:
import org.parboiled2.ParserInput
import org.parboiled2.Parser
import org.parboiled2.CharPredicate
sealed trait Token
case class TokenPart(tokenName : String) extends Token
case class StringToken(text: String) extends Token
// I moved pre-evaluated char predicates to the companion
// you may leave them inside the class if you want.
// I also moved literals like startToken and endToken here
object TokenExtractor {
val AlphaChar = CharPredicate.Alpha
val AlphaNum = CharPredicate.AlphaNum
val startToken = "{"
val endToken = "}"
}
class TokenExtractor(val input: ParserInput) extends Parser {
import TokenExtractor._
// may be you wanted zero or more? Anyway in this case
// shortcut can play nice here. In fact, if you want to stick
// with oneOrMore you can user AlphaNum.+ instead
def TokenName = rule {
AlphaChar ~ AlphaNum.*
}
// There's a shortcut for Extraction syntax. If you are extracting
// data to the case class and Rule arguments match the number of
// items in the case class's apply method
// you can simply give a name of this case class:
// the extraction operator '~>' should be located at the end of the
// from the official documtation:
// https://github.com/sirthias/parboiled2
// One more very useful feature is special support for
// case class instance creation:
//
// case class Person(name: String, age: Int)
// (foo: Rule2[String, Int]) ~> Person
//
def Token = rule {
startToken ~ capture(TokenName) ~ endToken ~> TokenPart
}
// the text should follow until the parser will meet the
// enclosing '{' character. Disclosing is not mandatory :)
def Text = rule {
oneOrMore(noneOf(startToken))
}
// Here we're capturing a data that matches
// pre-defined rule (in our case Text)
def TextString = rule {
capture(Text) ~> StringToken
}
def TextPart = rule {
TextString | Token
}
// EOI is mandatory. Parser is greedy, so it tells the parser
// where parsing procedure must end, so please, add it at the
// end of the input
def InputLine = rule {
zeroOrMore(TextPart) ~ EOI
}
}
object Main {
def main(args: Array[String]) {
val example =
"Hello {token1} today's date is {token2} would you like to call {token3}"
// parser input can be string, so put it inside the constructor
val result = new TokenExtractor(example).InputLine.run()
println(result)
}
}
I have defined the following macros to get file, line and object/class from current location:
http://pastebin.com/UsNLemnK
Using SBT, I have defined two projects, in order to compile the macros first, then the actual project using these macros.
The purpose of these macros are to be be used in a log method:
def log( msg: Any, srcFile: String = "", srcLine: String = "", srcClass:String = "")
I am then using this log method as follows:
log(msg, s"$F_",s"$L_",s"$C_")
where F_, L_ and C_ are defined in the macro.
Now, I would like to create a shortcut to avoid this boilerplate and just call:
log(msg)
which should automatically be replaced by
log(msg, s"$F_",s"$L_",s"$C_")
I could define a macro to do this:
def log_(msg: String) : Unit = macro logImpl
def logImpl( c: Context )(msg: c.Expr[String]): c.Expr[Unit] = {
import c.universe._
reify( log(msg.splice, srcFile=s"$F_", srcLine=s"$L_", srcClass=s"$C_") )
}
but again, this macro needs to be compiled before the project, where the log function itself is defined... So I don't see how to solve the compilation dependencies cycle...
Any suggestion about how to do this?
Thanks
Barring the use of macro annotations (which would necessarily and significantly alter your API's syntax), the problem you have to face is that you need the type-checked identifier of your log function.
Since you can't import the entire log implementation, a solution would be to:
wrap the method into a trait,
define this trait in the "macro" project,
add an implicit parameter to the log_ method,
in your "main" project, create an implementation of this trait, and instantiate this implementation in an implicit val visible everywhere you'd like to use the log_ macro (in the package object for example).
Of course, you could also use a simple FunctionN here and avoid the trait definition and implementation, but this way you'll avoid potential conflicts with other same-typed implicits.
In general, your code would resemble the following:
//"macro" project
trait EncapsulatingTrait {
def yourMethod(...)
}
object Macros {
def myMacro(...)(implicit param: EncapsulatingTrait) = macro myMacroImpl
def myMacroImpl( c: Context )(...)
(param: c.Expr[EncapsulatingTrait]): c.Expr[...] = {
import c.universe._
reify(param.splice.yourMethod(...))
}
}
//--------------------------
//"main" project
class Impl extends EncapsulatingTrait {
def yourMethod(...)
}
...
implicit val defaultParam = new Impl
import Macros.myMacro
myMacro(...)
In your specific case, here's how an implementation could look like:
//"macro" project
package yourpackage
import java.io.File
import language.experimental.macros
import scala.reflect.macros.Context
trait LogFunction {
def log( msg: Any, srcFile: String = "", srcLine: Int = -1, srcClass:String = "")
}
object Macros {
// get current line in source code
def L_ : Int = macro lineImpl
def lineImpl( c: Context ): c.Expr[Int] = {
import c.universe._
val line = Literal( Constant( c.enclosingPosition.line ) )
c.Expr[Int]( line )
}
// get current file from source code (relative path)
def F_ : String = macro fileImpl
def fileImpl( c: Context ): c.Expr[String] = {
import c.universe._
val absolute = c.enclosingPosition.source.file.file.toURI
val base = new File( "." ).toURI
val path = Literal( Constant( c.enclosingPosition.source.file.file.getName() ) )
c.Expr[String]( path )
}
// get current class/object (a bit sketchy)
def C_ : String = macro classImpl
def classImpl( c: Context ): c.Expr[String] = {
import c.universe._
val class_ = Literal( Constant( c.enclosingClass.toString.split(" ")( 1 ) ) )
c.Expr[String]( class_ )
}
def log_(msg: String)(implicit logFunc: LogFunction) : Unit = macro logImpl
def logImpl( c: Context )(msg: c.Expr[String])(logFunc: c.Expr[LogFunction]): c.Expr[Unit] = {
import c.universe._
reify( logFunc.splice.log(msg.splice, srcFile=fileImpl(c).splice, srcLine=lineImpl(c).splice, srcClass=classImpl(c).splice) )
}
}
//--------------------------
//"main" project
import yourpackage.LogFunction
class LogImpl extends LogFunction {
def log( msg: Any, srcFile: String = "", srcLine: Int = -1, srcClass:String = "") {
println(List(msg,srcFile,srcLine,srcClass).mkString("|"))
}
}
object testLog {
def main(args: Array[String]): Unit = {
implicit val defaultLog = new LogImpl
import yourpackage.Macros.log_
log_("blah")
}
}
(note that I had to correct the signature of log_ and tweak the macro call a bit)
I have an external process that I would like to treat as a
function from String=>String. Given a line of input, it will respond with a single line of output. It seems that I should use
scala.sys.process, which is clearly an elegant library that makes many
shell operations easily accessible from within scala. However, I
can't figure out how to perform this simple use case.
If I write a single line to the process' stdin, it prints the result
in a single line. How can I use sys.process to create a wrapper so I
can use the process interactively? For example, if I had an
implementation for ProcessWrapper, here is a program and it's output:
// abstract definition
class ProcessWrapper(executable: String) {
def apply(line: String): String
}
// program using an implementation
val process = new ProcessWrapper("cat -b")
println(process("foo"))
println(process("bar"))
println(process("baz"))
Output:
1 foo
2 bar
3 baz
It is important that the process is not reloaded for each call to process because there is a significant initialization step.
So - after my comment - this would be my solution
import java.io.BufferedReader
import java.io.File
import java.io.InputStream
import java.io.InputStreamReader
import scala.annotation.tailrec
class ProcessWrapper(cmdLine: String, lineListenerOut: String => Unit, lineListenerErr: String => Unit,
finishHandler: => Unit,
lineMode: Boolean = true, envp: Array[String] = null, dir: File = null) {
class StreamRunnable(val stream: InputStream, listener: String => Unit) extends Runnable {
def run() {
try {
val in = new BufferedReader(new InputStreamReader(this.stream));
#tailrec
def readLines {
val line = in.readLine
if (line != null) {
listener(line)
readLines
}
}
readLines
}
finally {
this.stream.close
finishHandler
}
}
}
val process = Runtime.getRuntime().exec(cmdLine, envp, dir);
val outThread = new Thread(new StreamRunnable(process.getInputStream, lineListenerOut), "StreamHandlerOut")
val errThread = new Thread(new StreamRunnable(process.getErrorStream, lineListenerErr), "StreamHandlerErr")
val sendToProcess = process.getOutputStream
outThread.start
errThread.start
def apply(txt: String) {
sendToProcess.write(txt.getBytes)
if (lineMode)
sendToProcess.write('\n')
sendToProcess.flush
}
}
object ProcessWrapper {
def main(args: Array[String]) {
val process = new ProcessWrapper("python -i", txt => println("py> " + txt),
err => System.err.println("py err> " + err), System.exit(0))
while (true) {
process(readLine)
}
}
}
The main part is the StreamRunnable, where the process is read in a thread and the received line is passed on to a "LineListener" (a simple String => Unit - function).
The main is just a sample implementation - calling python ;)
I'm not sure, but you want somethings like that ?
case class ProcessWrapper(executable: String) {
import java.io.ByteArrayOutputStream
import scala.concurrent.duration.Duration
import java.util.concurrent.TimeUnit
lazy val process = sys.runtime.exec(executable)
def apply(line: String, blockedRead: Boolean = true): String = {
process.getOutputStream().write(line.getBytes())
process.getOutputStream().flush()
val r = new ByteArrayOutputStream
if (blockedRead) {
r.write(process.getInputStream().read())
}
while (process.getInputStream().available() > 0) {
r.write(process.getInputStream().read())
}
r.toString()
}
def close() = process.destroy()
}
val process = ProcessWrapper("cat -b")
println(process("foo\n"))
println(process("bar\n"))
println(process("baz\n"))
println(process("buz\n"))
println(process("puz\n"))
process.close
Result :
1 foo
2 bar
3 baz
4 buz
5 puz
I think that PlayCLI is a better way.
http://blog.greweb.fr/2013/01/playcli-play-iteratees-unix-pipe/ came across this today and looks exactly like what you want
How about using an Akka actor. The actor can have state and thus a reference to an open program (in a thread). You can send messages to that actor.
ProcessWrapper might be a typed actor itself or just something that converts the calls of a function to a call of an actor. If you only have 'process' as method name, then wrapper ! "message" would be enough.
Having a program open and ready to receive commands sounds like an actor that receives messages.
Edit: Probably I got the requirements wrong. You want to send multiple lines to the same process. That's not possible with the below solution.
One possibility would be to add an extension method to the ProcessBuilder that allows for taking the input from a string:
implicit class ProcessBuilderWithStringInput(val builder: ProcessBuilder) extends AnyVal {
// TODO: could use an implicit for the character set
def #<<(s: String) = builder.#<(new ByteArrayInputStream(s.getBytes))
}
You can now use the method like this:
scala> ("bc":ProcessBuilder).#<<("3 + 4\n").!!
res9: String =
"7
"
Note that the type annotation is necessary, because we need two conversions (String -> ProcessBuilder -> ProcessBuilderWithStringInput, and Scala will only apply one conversion automatically.