I am using 'scalafmt' command to ensure that I've no formatting errors in my Scala code. It keeps failing with this error message:
Looking for unformatted files... (98.35 %, 238 / 242)
error: --test failed
Those are the last two lines. There's no other error message in the log. Is there a configuration I can use that will give me more information about this failure?
By default, Scalafmt errors are reported to System.err. Extend org.scalafmt.interfaces.ScalafmtReporter to customize error reporting to handle parse and config errors.
class ScalafmtErrReport(out: PrintStream) extends ScalafmtReporter {
override def error(file: Path, e: Throwable): Unit = {
out.print(s"error: $file: ")
trimStacktrace(e)
e.printStackTrace(out)
}
override def error(path: Path, message: String): Unit = {
out.println(s"error: $path: $message")
}
override def error(file: Path, message: String, e: Throwable): Unit = {
error(file, ScalafmtException(message, e))
}
}
check: https://scalameta.org/scalafmt/docs/installation.html
Related
I have found that scala compiler report compile error differently depends on whether it is used in compile-time or run-time.
E.g. for a simple implicit not found case, in compile time the report is:
newSource1.scala:6: error: type mismatch;
L|R
f(new L)
^
In runtime, when the same code was evaluated and the error message was captured directly:
class TestCase(code: String, extra: String) {
def toolbox(extra: String): ToolBox[universe.type] = {
ToolBox(cm).mkToolBox(options = s"$opts $extra")
}
def compile(): Any = {
import SpecHelpers._
val tb = toolbox(extra)
tb.eval(tb.parse(code))
}
def compileError(): String =
Try(compile()) match {
case Failure(ee) =>
ee match {
case te: ToolBoxError =>
te.message.linesIterator.toList.drop(2).mkString("\n")
case t =>
throw t
}
case Success(_) =>
sys.error("compiling succeeded")
}
}
the line number and location cursor is omitted:;
implicit error;
!I e: F[Arg]
Is there a way to add the missing part back?
You can specify frontEnd for ToolBox (default frontEnd is mkSilentFrontEnd())
val tb = cm.mkToolBox(
frontEnd = new FrontEnd {
override def display(info: Info): Unit = println(info)
},
options = "-Xlog-implicits -Vimplicits -Vimplicits-verbose-tree -d out"
// with `-d out` toolbox keeps `.class` files in directory `out`
// rather than in memory, so you can decompile `.class` files
// if you want to investigate auto-generated code
)
tb.compile(tb.parse(
"""implicit val i: Int = 1
|implicitly[String]""".stripMargin))
//Info(source-<toolbox>,line-2,offset=34,implicit error;
//!I e: String,ERROR)
//Exception in thread "main" java.lang.ExceptionInInitializerError
// at App.main(App.scala)
//Caused by: scala.tools.reflect.ToolBoxError: reflective compilation has failed:
//
//implicit error;
//!I e: String
// at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.throwIfErrors(ToolBoxFactory.scala:332)
// at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.wrapInPackageAndCompile(ToolBoxFactory.scala:214)
// at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.compile(ToolBoxFactory.scala:268)
// at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.$anonfun$compile$13(ToolBoxFactory.scala:445)
// at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$withCompilerApi$.apply(ToolBoxFactory.scala:371)
// at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.compile(ToolBoxFactory.scala:438)
// at App$.<clinit>(App.scala:20)
// ... 1 more
I am trying to implement Apache Beam in Scala. To do that, I took a simple task of loading a file from my local(windows) and getting a word count.
This is what I wrote.
object ReadFromFile {
def main(args: Array[String]): Unit = {
PipelineOptionsFactory.register(Class[MyOptions])
val options = PipelineOptionsFactory.fromArgs(args: _*).withValidation() .as(classOf[MyOptions])
val pipeline = Pipeline.create(options)
pipeline.apply("ReadFiles", TextIO.read().from(options.getInputFile))
.apply(ParDo.of(new ExtractWords))
.apply(Count.perElement())
.apply(MapElements.via(new FormatResult))
.apply("WriteWords", TextIO.write().to(options.getOutput))
pipeline.run().waitUntilFinish()
}
}
This is the rest of my setup. I followed the beam documentation from here to create an interface to implement get & set methods to read input file and create an output file.
trait MyOptions extends PipelineOptions{
#Description("Path of the file to read from")
#Default.String("path_of_input_file.txt")
def getInputFile: String
def setInputFile(path: String)
#Description("Path of the file to write to")
#Required
def getOutput: String
def setOutput(path: String)
}
and other UDFs to load, split & count the records.
class ExtractWords extends DoFn[String, String] {
#ProcessElement
def processElement(c: ProcessContext): Unit = {
for (word <- c.element().split(",")) yield {
if (word.nonEmpty) c.output(word)
}
}
}
class FormatResult extends SimpleFunction[KV[String, java.lang.Long], String] {
override def apply(input: KV[String, java.lang.Long]): String = {
input.getKey + ": " + input.getValue
}
}
class CountWords extends PTransform[PCollection[String], PCollection[KV[String, java.lang.Long]]] {
override def expand(input: PCollection[String]): PCollection[KV[String, lang.Long]] = {
input.apply(ParDo.of(new ExtractWords)) //Ignore IntelliJ error: "Cannot resolve apply". The code will compile.
.apply(Count.perElement())
}
}
But I am seeing a compilation error in my main class on the apply method at getting the count: .apply(Count.perElement())
which says:
Cannot resolve overloaded method 'apply'
This is my project structure and the same error could be seen in the image:
This is the first time I am learning Apache Beam. Could anyone let me know what is the mistake I did here and how can I fix the error ?
How can I capture the exception stacktrace into a file in Scala? I tried with System.setErr but it didn't work, while Console.withErr can be set only for a code snippet and not for the whole app. Any idea?
I would like to log to a file all the uncaught exceptions of the app
Edit:
A bit more context: I run inside a spark-shell with a jar added in the classpath. What I do is instantiating an object of a class and the first code ran inside the class is the following, which doesn't give me the
expected print
Thread.currentThread().setUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
def uncaughtException(t: Thread, e: Throwable): Unit = {
logger.error("exception logged")
println("exception logged")
}
})
throw new Exception("my ex")
but I don't see the print neither in stdout nor in the log file
Just use the Thread.setUncaughtExceptionHandler:
Java uncaught global exception handler
Use Thread.currentThread() if your app is single-threaded to get the current thread.
Edit
object ErrorHandling {
def main(args: Array[String]): Unit = {
Thread.currentThread().setUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
def uncaughtException(t: Thread, e: Throwable): Unit = {
// logger.error("exception logged")
println(s"exception logged: $e")
e.printStackTrace()
}
})
throw new Exception(s"boom")
}
}
Should give you something like:
exception logged: java.lang.Exception: boom
java.lang.Exception: boom
at ErrorHandling$.main(ErrorHandling.scala:10)
at ErrorHandling.main(ErrorHandling.scala)
After upgrading my scala to latest version i got that error:
type mismatch; found : org.specs2.execute.Failure required: T
My code:
def shouldThrow[T <: Exception](exClazz: Class[T])(body: => Unit): T = {
try {
body
} catch {
case e: Throwable =>
if (e.getClass == exClazz) return e.asInstanceOf[T]
val failure = new Failure("Expected %s but got %s".format(exClazz, e.getClass), "", new Exception().getStackTrace.toList, org.specs2.execute.NoDetails())
val rethrown = new FailureException(failure)
rethrown.initCause(e)
throw rethrown
}
failure("Exception expected, but has not been thrown")
}
I got this error at last line failure("...")
Any idea whats goin on?
failure in mutable specifications used to be defined without a return type and then it return type was Nothing. This has been fixed to return Failure and be more consistent with non-mutable specifications.
However you don't need failure in your code because there are exceptions matchers (see the "Exception" tab here) and you can write:
body must throwAn[IllegalArgumentException]
As the message says, method failure returns a Failure, but you tell the compiler you want to return T. You can just throw an exception in the last line (which failure probably used to do). You can also avoid passing Class[T] manually by using ClassTag:
def shouldThrow[T <: Exception](body: => Unit)(implicit ct: ClassTag[T]): T = {
... // as before
case e: Throwable =>
val exClass = ct.runtimeClass
... // as before
}
// usage
shouldThrow[IOException] {
// body
}
def buildLog(appName: String) = {
val file = s"${sys.env("HOME")}/builds/logs/${appName}.log"
val f = new File(file)
Enumerator.fromFile(f).map(new String(_))
}
I want to display a rolling log file to the web page, I use Enumerator.fromFile() method and map the byte array to String, but I am getting some strange chars like
From xxx:yyy/zzz
* branch master -> FETCH_HEAD
Already up-to-date.
[0m[[0minfo[0m] [0mLoading project definition from /home/zzz/builds/sources/ops-ui/project[0m
[0m[[0minfo[0m] [0mSet current project to zzz (in build file:/home/zzz/builds/sources/ops-ui/)[0m
[0m[[32msuccess[0m] [0mTotal time: 0 s, completed 2013-12-30 17:38:53[0m
How to display the content correctly?
Those characters define colors the console should use to display the text (e.g. info in cyan and errors in red).
To remove the characters from your file, you can change the logging configuration, for example replace %coloredLevel with %level in conf/logger.xml.
Alternatively, you could implement a custom LogManager to rewrite logs as needed. This is inspired by similar emojiLogs, but I did not test it:
logManager ~= { lm =>
new LogManager {
def apply(
data: sbt.Settings[Scope],
state: State,
task: Def.ScopedKey[_],
writer: java.io.PrintWriter
) = {
val l = lm.apply(data, state, task, writer)
new Logger {
val Escapes = Set("M\\\\[2K", "\\[0m", "\\[32m")
def ommitColorChars(str: String) = Escapes.fold(str)(_.replaceAll(_))
def log(level: Level.Value, message: => String) =
l.log(level, ommitColorChars(message))
def success(message: => String) = l.success(message)
def trace(t: => Throwable) = l.trace(t)
override def ansiCodesSupported = l.ansiCodesSupported
}
}
}
}
Just to complete a bit more the answer, I found a more general way of removing coloured logs. The colouring characters can be removed as (assuming you have a string with the coloured logs):
string.replaceAll("""\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]""", "")
Inspired by the equivalent "sed" in Linux command (1):
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"