I have just started to look at ZIO to try to improve my Scala. I have started by trying to update some of my old code. I have wrapped some legacy code which returns a configuration wrapped in an option and I'm converting that to a ZIO which I'm then using in for comprehension.
The code works as expected but if I return None I get:
Fiber failed.
A checked error was not handled.
None
Fiber:Id(1612612180323,1) was supposed to continue to:
a future continuation at tryzio.MyApp$.run(MyApp.scala:94)
a future continuation at zio.ZIO.exitCode(ZIO.scala:543)
Fiber:Id(1612612180323,1) execution trace:
at zio.ZIO$.fromOption(ZIO.scala:3246)
at tryzio.MyApp$.run(MyApp.scala:93)
The code works as expected for both a Some and a None, but I get the spurious Fibre messages for a None.
The code that generates this is really very simple:
def run(args: List[String]) = ({
for {
cmdln <- ZIO.fromOption(getConfig(args.toArray))
_ <- putStrLn("Model Data Builder")
_ <- putStrLn(s"\nVerbose: ${cmdln.verbose}")
} yield()
}).exitCode
But I have clearly missed something obvious! I'm new to ZIO so please use small words when explains my lack of understanding. Do I need to join all Fibres? I have tried to catch the None and then exit but actually my code never strops running if I do that - Very strange.
.exitCode caught an error (empty configuration) which hasn't been handled in the code, printed debug information in stdErr and exited the program with status 1. So it works as expected.
I agree that the error message is a bit misleading and should start with something something business-related rather than fiber failed.
You might fire a ticket on github.
I am playing around with calling external command from Scala. Here is a stripped out example of what I am working on:
import scala.sys.process._
object Testing {
def main(args: Array[String]) {
val command = "ls"
val result = command!
if (result != 0) { // <---- illegal start of simple expression
println("Error")
return
}
}
}
I am getting a compile error: illegal start of simple expression for the line with the if statement. I can fix it with a new line:
val result = command!
// Add a line
if (result != 0) {
My suspicion is that it has something to do with the ! postfix function, but it was my understanding that superfluous lines/whitespaces shouldn't make a difference to the compiler.
You need to explicitly enable postfix expressions:
1) Importing the flag locally: import scala.language.postfixOps
2) or adding the flag to the project itself: scalacOptions += "-language:postfixOps"
The above link in the comment from #Łukasz contains lots of info about this feature. Also, see http://docs.scala-lang.org/style/method-invocation.html in the "Suffix Notation" section for your exact use case.
EDIT: maybe it was not clear enough, but as #Łukasz pointed in comments, importing/enabling postfix expressions doesn't make your code compile. It just avoids the compiler warning. Your code won't compile because the semicolons are optional and the compiler is treating the ! operator as infix, and thus taking elements from the next line for the expressions. This is exactly what the documentation in the link above states with exactly this same example:
This style is unsafe, and should not be used. Since semicolons are
optional, the compiler will attempt to treat it as an infix method if
it can, potentially taking a term from the next line.
names toList
val answer = 42 // will not compile!
This may result in unexpected compile errors at best, and happily
compiled faulty code at worst. Although the syntax is used by some
DSLs, it should be considered deprecated, and avoided.
In scalatest using FunSpec I have some code that fires in the afterEach. I would like to execute some code to get a screenshot only when the test fails. Just about everything I have looked at tries to solve this by putting asserts in try blocks which seems awful. Is there anything like onTestFailure in TestNG or a context I can get like RSpec to determine if the test failed? Looking at the scala doc I see a implementation that takes a configMap, but that is empty when I run the test. Any help would be appreciated.
pretty sure I figured it out. coming from a TestNG background it seemed weird to have to mess with fixtures to accomplish it. I suspect others with a background like mine may also look in all the wrong places as well, so going to leave this here to help others:
override def withFixture(test: NoArgTest) = { // after
val outcome = super.withFixture(test)
outcome match {
case Failed(ex) =>
// log ex (the exception) and a screenshot
}
outcome
}
So, I have the following kind of code running in each map tasks on Spark.
#volatile var res = (someProgram + fileName) !
var cmdRes = ("rm " + fileName) !;
The filenames for each map tasks are unique. The basic idea is that once the first command finishes, the second command deletes the file. However, I notice that the program sometimes complain that the file does not exist. It seems that the subprocess call is not synchronous, that is, it does not wait for the subprocess to complete. Is that correct. And if it indeed the case, how can we correct that?
As you can see in the docs, the ! method blocks until exit. The docs say that it:
Starts the process represented by this builder, blocks until it exits, and returns the exit code.
It's possible that you should be checking the exit code to interpret the result and to deal with exceptional cases.
When creating process commands by concatenation, you are often better off using the Seq extensions (as opposed to the String ones) to create a ProcessBuilder. The docs even include this helper, which might help you:
// This uses ! to get the exit code
def fileExists(name: String) = Seq("test", "-f", name).! == 0
I just had a look at the new scala.sys and scala.sys.process packages to see if there is something helpful here. However, I am at a complete loss.
Has anybody got an example on how to actually start a process?
And, which is most interesting for me: Can you detach processes?
A detached process will continue to run when the parent process ends and is one of the weak spots of Ant.
UPDATE:
There seem to be some confusion what detach is. Have a real live example from my current project. Once with z-Shell and once with TakeCommand:
Z-Shell:
if ! ztcp localhost 5554; then
echo "[ZSH] Start emulator"
emulator \
-avd Nexus-One \
-no-boot-anim \
1>~/Library/Logs/${PROJECT_NAME}-${0:t:r}.out \
2>~/Library/Logs/${PROJECT_NAME}-${0:t:r}.err &
disown
else
ztcp -c "${REPLY}"
fi;
Take-Command:
IFF %#Connect[localhost 5554] lt 0 THEN
ECHO [TCC] Start emulator
DETACH emulator -avd Nexus-One -no-boot-anim
ENDIFF
In both cases it is fire and forget, the emulator is started and will continue to run even after the script has ended. Of course having to write the scripts twice is a waste. So I look into Scala now for unified process handling without cygwin or xml syntax.
First import:
import scala.sys.process.Process
then create a ProcessBuilder
val pb = Process("""ipconfig.exe""")
Then you have two options:
run and block until the process exits
val exitCode = pb.!
run the process in background (detached) and get a Process instance
val p = pb.run
Then you can get the exitcode from the process with (If the process is still running it blocks until it exits)
val exitCode = p.exitValue
If you want to handle the input and output of the process you can use ProcessIO:
import scala.sys.process.ProcessIO
val pio = new ProcessIO(_ => (),
stdout => scala.io.Source.fromInputStream(stdout)
.getLines.foreach(println),
_ => ())
pb.run(pio)
I'm pretty sure detached processes work just fine, considering that you have to explicitly wait for it to exit, and you need to use threads to babysit the stdout and stderr. This is pretty basic, but it's what I've been using:
/** Run a command, collecting the stdout, stderr and exit status */
def run(in: String): (List[String], List[String], Int) = {
val qb = Process(in)
var out = List[String]()
var err = List[String]()
val exit = qb ! ProcessLogger((s) => out ::= s, (s) => err ::= s)
(out.reverse, err.reverse, exit)
}
Process was imported from SBT. Here's a thorough guide on how to use the process library as it appears in SBT.
https://github.com/harrah/xsbt/wiki/Process
Has anybody got an example on how to
actually start a process?
import sys.process._ // Package object with implicits!
"ls"!
And, which is most interesting for me:
Can you detach processes?
"/path/to/script.sh".run()
Most of what you'll do is related to sys.process.ProcessBuilder, the trait. Get to know that.
There are implicits that make usage less verbose, and they are available through the package object sys.process. Import its contents, like shown in the examples. Also, take a look at its scaladoc as well.
The following function will allow easy use if here documents:
def #<<< (command: String) (hereDoc: String) =
{
val process = Process (command)
val io = new ProcessIO (
in => {in.write (hereDoc getBytes "UTF-8"); in.close},
out => {scala.io.Source.fromInputStream(out).getLines.foreach(println)},
err => {scala.io.Source.fromInputStream(err).getLines.foreach(println)})
process run io
}
Sadly I was not able to (did not have the time to) to make it an infix operation. Suggested calling convention is therefore:
#<<< ("command") {"""
Here Document data
"""}
It would be call if anybody could give me a hint on how to make it a more shell like call:
"command" #<<< """
Here Document data
""" !
Documenting process a little better was second on my list for probably two months. You can infer my list from the fact that I never got to it. Unlike most things I don't do, this is something I said I'd do, so I greatly regret that it remains as undocumented as it was when it arrived. Sword, ready yourself! I fall upon thee!
If I understand the dialog so far, one aspect of the original question is not yet answered:
how to "detach" a spawned process so it continues to run independently of the parent scala script
The primary difficulty is that all of the classes involved in spawning a process must run on the JVM, and they are unavoidably terminated when the JVM exits. However, a workaround is to indirectly achieve the goal by leveraging the shell to do the "detach" on your behalf. The following scala script, which launches the gvim editor, appears to work as desired:
val cmd = List(
"scala",
"-e",
"""import scala.sys.process._ ; "gvim".run ; System.exit(0);"""
)
val proc = cmd.run
It assumes that scala is in the PATH, and it does (unavoidably) leave a JVM parent process running as well.