import scala.io._
object Sum {
def main(args :Array[String]):Unit = {
println("Enter some numbers and press ctrl-c")
val input = Source.fromInputStream(System.in)
val lines = input.getLines.toList
println("Sum "+sum(lines))
}
def toInt(in:String):Option[Int] =
try{
Some(Integer.parseInt(in.trim))
}
catch {
case e: NumberFormatException => None
}
def sum(in :Seq[String]) = {
val ints = in.flatMap(s=>toInt(s))
ints.foldLeft(0) ((a,b) => a +b)
} }
I am trying to run this program after passing input I have press
ctrl + c but
It gives this message E:\Scala>scala HelloWord.scala Enter some
numbers and press ctrl-c 1 2 3 Terminate batch job (Y/N)?
Additional observations, note trait App to make an object executable, hence not having to declare a main(...) function, for instance like this,
object Sum extends App {
import scala.io._
import scala.util._
val nums = Source.stdin.getLines.flatMap(v => Try(v.toInt).toOption)
println(s"Sum: ${nums.sum}")
}
Using Try, non successful conversions from String to Int are turned to None and flattened out.
Also note objects and classes are capitalized, hence instead of object sum by convention we write object Sum.
You can also use an external API. I really like scallop API
Try this piece of code. It should work as intended.
object Sum {
def main(args: Array[String]) {
val lines = io.Source.stdin.getLines
val numbers = lines.map(_.toInt)
println(s"Sum: ${numbers.sum}")
}
}
Plus, the correct shortcut to end the input stream is Ctrl + D.
Related
I'm beginning with Scala. I have a program which have a method with a while loop which run until the program is not ended.
But for my test, I need to execute this method only once (or twice). In java, I would have used a mutable variable that I would have decremented in order to stop my treatment.
Maybe a condition inside my while loop that I override for my test.
def receive = {
val iterator = stream.iterator()
while (iterator.hasNext && my_condition()) {
something_to_do
}
}
I know it's a stupid question, but could you please advice me ?
Try:
iterator.takeWhile(my_condition).foreach(something_to_do)
or:
iterator.take(n).foreach(something_to_do)
if you just want the first n entries.
Or, if something_to_do returns a result (rather than Unit), and you want to return an iterator of those results, you can use:
iterator.takeWhile(my_condition).map(something_to_do)
(or .take(n).map(...) )
Consider this for comprehension,
for (_ <- iterator if my_condition()) something_to_do
where each iterated value is ignored (note _) and the todo part is invoked while the condition holds.
I think an approach like the following is acceptable:
import akka.actor.{Props, Actor}
import scala.io.Source
object TestableActor {
def props = Props(new TestableActor())
def testProps = Props(new TestableActor(true))
case class Message(stream: Stream)
}
class TestableActor(doOnce: Boolean = false) extends Actor {
import TestableActor._
val stream: Stream = ???
def receive = {
case Message(stream) =>
val iterator = stream.iterator
if(doOnce) {
something_to_do
} else {
while (iterator.hasNext && my_condition()) {
something_to_do
}
}
}
def my_condition(): Boolean = ???
def something_to_do: Unit = ???
}
In your production code, use
context.actorOf(TestableActor.props)
In your test use
TestActorRef[TestableActor](TestableActor.testProps)
I have an Iterable[String] and I want to stream that to an external Process and return an Iterable[String] for the output.
I feel like this should work as it compiles
import scala.sys.process._
object PipeUtils {
implicit class IteratorStream(s: TraversableOnce[String]) {
def pipe(cmd: String) = s.toStream.#>(cmd).lines
def run(cmd: String) = s.toStream.#>(cmd).!
}
}
However, Scala tries to execute the contents of s instead of pass them in to standard in. Can anyone please tell me what I'm doing wrong?
UPDATE:
I think that my original problem was that the s.toStream was being implicity converted to a ProcessBuilder and then executed. This is incorrect as it's the input to the process.
I have come up with the following solution. This feels very hacky and wrong but it seems to work for now. I'm not writing this as an answer because I feel like the answer should be one line and not this gigantic thing.
object PipeUtils {
/**
* This class feels wrong. I think that for the pipe command it actually loads all of the output
* into memory. This could blow up the machine if used wrong, however, I cannot figure out how to get it to
* work properly. Hopefully http://stackoverflow.com/questions/28095469/stream-input-to-external-process-in-scala
* will get some good responses.
* #param s
*/
implicit class IteratorStream(s: TraversableOnce[String]) {
val in = (in: OutputStream) => {
s.foreach(x => in.write((x + "\n").getBytes))
in.close
}
def pipe(cmd: String) = {
val output = ListBuffer[String]()
val io = new ProcessIO(in,
out => {Source.fromInputStream(out).getLines.foreach(output += _)},
err => {Source.fromInputStream(err).getLines.foreach(println)})
cmd.run(io).exitValue
output.toIterable
}
def run(cmd: String) = {
cmd.run(BasicIO.standard(in)).exitValue
}
}
}
EDIT
The motivation for this comes from using Spark's .pipe function on an RDD. I want this exact same functionality on my local code.
Assuming scala 2.11+, you should use lineStream as suggested by #edi. The reason is that you get a streaming response as it becomes available instead of a batched response. Let's say I have a shell script echo-sleep.sh:
#/usr/bin/env bash
# echo-sleep.sh
while read line; do echo $line; sleep 1; done
and we want to call it from scala using code like the following:
import scala.sys.process._
import scala.language.postfixOps
import java.io.ByteArrayInputStream
implicit class X(in: TraversableOnce[String]) {
// Don't do the BAOS construction in real code. Just for illustration.
def pipe(cmd: String) =
cmd #< new ByteArrayInputStream(in.mkString("\n").getBytes) lineStream
}
Then if we do a final call like:
1 to 10 map (_.toString) pipe "echo-sleep.sh" foreach println
a number in the sequence appears on STDOUT every 1 second. If you buffer, and convert to an Iterable as in your example, you will lose this responsiveness.
Here's a solution demonstrating how to write the process code so that it streams both the input and output. The key is to produce a java.io.PipedInputStream that is passed to the input of the process. This stream is filled from the iterator asynchronously via a java.io.PipedOutputStream. Obviously, feel free to change the input type of the implicit class to an Iterable.
Here's an iterator used to show this works.
/**
* An iterator with pauses used to illustrate data streaming to the process to be run.
*/
class PausingIterator[A](zero: A, until: A, pauseMs: Int)(subsequent: A => A)
extends Iterator[A] {
private[this] var current = zero
def hasNext = current != until
def next(): A = {
if (!hasNext) throw new NoSuchElementException
val r = current
current = subsequent(current)
Thread.sleep(pauseMs)
r
}
}
Here's the actual code you want
import java.io.PipedOutputStream
import java.io.PipedInputStream
import java.io.InputStream
import java.io.PrintWriter
// For process stuff
import scala.sys.process._
import scala.language.postfixOps
// For asynchronous stream writing.
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
/**
* A streaming version of the original class. This does not block to wait for the entire
* input or output to be constructed. This allows the process to get data ASAP and allows
* the process to return information back to the scala environment ASAP.
*
* NOTE: Don't forget about error handling in the final production code.
*/
implicit class X(it: Iterator[String]) {
def pipe(cmd: String) = cmd #< iter2is(it) lineStream
/**
* Convert an iterator to an InputStream for use in the pipe function.
* #param it an iterator to convert
*/
private[this] def iter2is[A](it: Iterator[A]): InputStream = {
// What is written to the output stream will appear in the input stream.
val pos = new PipedOutputStream
val pis = new PipedInputStream(pos)
val w = new PrintWriter(pos, true)
// Scala 2.11 (scala 2.10, use 'future'). Executes asynchrously.
// Fill the stream, then close.
Future {
it foreach w.println
w.close
}
// Return possibly before pis is fully written to.
pis
}
}
The final call will show display 0 through 9 and will pause for 3 seconds in between the displaying of each number (second pause on the scala side, 1 second pause on the shell script side).
// echo-sleep.sh is the same script as in my previous post
new PausingIterator(0, 10, 2000)(_ + 1)
.map(_.toString)
.pipe("echo-sleep.sh")
.foreach(println)
Output
0 [ pause 3 secs ]
1 [ pause 3 secs ]
...
8 [ pause 3 secs ]
9 [ pause 3 secs ]
I'm looking to create a way to dynamically call logic depending on template id within scala. So template id 1 calls logic a, template id 2 call logic b, etc. The logic will be diverse but will have the same inputs/outputs. Also the number of different template ids will get into the thousands and will not be known ahead of time, so a loose coupling feels the way to go.
I've started looking at reflection to do this using scala 2.11.1 and can statically use reflection when I know the logic to be used ahead of time but have not found the correct way to dynamically use reflection, so for example passing in template id 2 will call logic b.
Below is a cut down example showing how the static version works and the skeleton I have so far for the dynamic version.
package thePackage
import scala.reflect.runtime.{universe => ru}
trait theTrait { def theMethod(x: String): Unit }
// the different logic held in different objects
object object1 extends theTrait {
def theMethod(x: String) = { println("a " + x ) }
}
object object2 extends theTrait {
def theMethod(x: String) = { println("b " + x ) }
}
object object3 extends theTrait {
def theMethod(x: String) = { println("c " + x ) }
}
// run static/dynamic reflection methods
object ReflectionTest {
// "static" invocation calling object1.theMethod
def staticInvocation() = {
val m = ru.runtimeMirror(getClass.getClassLoader)
val im = m.reflect(thePackage.object1)
val method = ru.typeOf[thePackage.object1.type]
.decl(ru.TermName("theMethod")).asMethod
val methodRun = im.reflectMethod(method)
methodRun("test")
}
staticInvocation
// "dynamic" invocation using integer to call different methods
def dynamicInvocation( y: Integer) = {
val m = ru.runtimeMirror(getClass.getClassLoader)
val module = m.staticModule("thePackage.object" + y)
val im = m.reflectModule(module)
// stuck... static approach does not work here
}
dynamicInvocation(1)
dynamicInvocation(2)
dynamicInvocation(3)
}
What needs to be added/changed to the dynamicInvocation method to make this work, or should I be using a different approach?
You need to get an instance mirror for your module, on which you can reflect the method.
def dynamicInvocation( y: Integer) = {
val m = ru.runtimeMirror(getClass.getClassLoader)
val module = m.staticModule("thePackage.object" + y)
val im = m.reflectModule(module)
val method = im.symbol.info.decl(ru.TermName("theMethod")).asMethod
val objMirror = m.reflect(im.instance)
objMirror.reflectMethod(method)("test")
}
It seems that TermName method in above solution has been replaced by newTermName and also the info.decl seems to not work. Below line worked for me
val method = im.symbol.typeSignature.member(ru.newTermName("testMethod")).asMethod
I have an external process that I would like to treat as a
function from String=>String. Given a line of input, it will respond with a single line of output. It seems that I should use
scala.sys.process, which is clearly an elegant library that makes many
shell operations easily accessible from within scala. However, I
can't figure out how to perform this simple use case.
If I write a single line to the process' stdin, it prints the result
in a single line. How can I use sys.process to create a wrapper so I
can use the process interactively? For example, if I had an
implementation for ProcessWrapper, here is a program and it's output:
// abstract definition
class ProcessWrapper(executable: String) {
def apply(line: String): String
}
// program using an implementation
val process = new ProcessWrapper("cat -b")
println(process("foo"))
println(process("bar"))
println(process("baz"))
Output:
1 foo
2 bar
3 baz
It is important that the process is not reloaded for each call to process because there is a significant initialization step.
So - after my comment - this would be my solution
import java.io.BufferedReader
import java.io.File
import java.io.InputStream
import java.io.InputStreamReader
import scala.annotation.tailrec
class ProcessWrapper(cmdLine: String, lineListenerOut: String => Unit, lineListenerErr: String => Unit,
finishHandler: => Unit,
lineMode: Boolean = true, envp: Array[String] = null, dir: File = null) {
class StreamRunnable(val stream: InputStream, listener: String => Unit) extends Runnable {
def run() {
try {
val in = new BufferedReader(new InputStreamReader(this.stream));
#tailrec
def readLines {
val line = in.readLine
if (line != null) {
listener(line)
readLines
}
}
readLines
}
finally {
this.stream.close
finishHandler
}
}
}
val process = Runtime.getRuntime().exec(cmdLine, envp, dir);
val outThread = new Thread(new StreamRunnable(process.getInputStream, lineListenerOut), "StreamHandlerOut")
val errThread = new Thread(new StreamRunnable(process.getErrorStream, lineListenerErr), "StreamHandlerErr")
val sendToProcess = process.getOutputStream
outThread.start
errThread.start
def apply(txt: String) {
sendToProcess.write(txt.getBytes)
if (lineMode)
sendToProcess.write('\n')
sendToProcess.flush
}
}
object ProcessWrapper {
def main(args: Array[String]) {
val process = new ProcessWrapper("python -i", txt => println("py> " + txt),
err => System.err.println("py err> " + err), System.exit(0))
while (true) {
process(readLine)
}
}
}
The main part is the StreamRunnable, where the process is read in a thread and the received line is passed on to a "LineListener" (a simple String => Unit - function).
The main is just a sample implementation - calling python ;)
I'm not sure, but you want somethings like that ?
case class ProcessWrapper(executable: String) {
import java.io.ByteArrayOutputStream
import scala.concurrent.duration.Duration
import java.util.concurrent.TimeUnit
lazy val process = sys.runtime.exec(executable)
def apply(line: String, blockedRead: Boolean = true): String = {
process.getOutputStream().write(line.getBytes())
process.getOutputStream().flush()
val r = new ByteArrayOutputStream
if (blockedRead) {
r.write(process.getInputStream().read())
}
while (process.getInputStream().available() > 0) {
r.write(process.getInputStream().read())
}
r.toString()
}
def close() = process.destroy()
}
val process = ProcessWrapper("cat -b")
println(process("foo\n"))
println(process("bar\n"))
println(process("baz\n"))
println(process("buz\n"))
println(process("puz\n"))
process.close
Result :
1 foo
2 bar
3 baz
4 buz
5 puz
I think that PlayCLI is a better way.
http://blog.greweb.fr/2013/01/playcli-play-iteratees-unix-pipe/ came across this today and looks exactly like what you want
How about using an Akka actor. The actor can have state and thus a reference to an open program (in a thread). You can send messages to that actor.
ProcessWrapper might be a typed actor itself or just something that converts the calls of a function to a call of an actor. If you only have 'process' as method name, then wrapper ! "message" would be enough.
Having a program open and ready to receive commands sounds like an actor that receives messages.
Edit: Probably I got the requirements wrong. You want to send multiple lines to the same process. That's not possible with the below solution.
One possibility would be to add an extension method to the ProcessBuilder that allows for taking the input from a string:
implicit class ProcessBuilderWithStringInput(val builder: ProcessBuilder) extends AnyVal {
// TODO: could use an implicit for the character set
def #<<(s: String) = builder.#<(new ByteArrayInputStream(s.getBytes))
}
You can now use the method like this:
scala> ("bc":ProcessBuilder).#<<("3 + 4\n").!!
res9: String =
"7
"
Note that the type annotation is necessary, because we need two conversions (String -> ProcessBuilder -> ProcessBuilderWithStringInput, and Scala will only apply one conversion automatically.
trying to run the sample code in the Apress book called "Beginning Scala". I even downloaded the code from their website to make sure I didn't goof. Getting the following message:
/root/sum.scala:19: error: missing arguments for method collect in trait Iterator;
follow this method with `_' if you want to treat it as a partially applied function
val lines = input.getLines.collect
^
one error found
and here is the source code i used (running Scala version 2.8.1.final (Java HotSpot(TM) Server VM, Java 1.6.0_22 on Fedora 13)
import scala.io._
def toInt(in: String): Option[Int] =
try {
Some(Integer.parseInt(in.trim))
} catch {
case e: NumberFormatException => None
}
def sum(in: Seq[String]) = {
val ints = in.flatMap(s => toInt(s))
ints.foldLeft(0)((a, b) => a + b)
}
println("Enter some numbers and press ctrl-D (Unix/Mac) ctrl-C (Windows)")
val input = Source.fromInputStream(System.in)
val lines = input.getLines.collect
println("Sum "+sum(lines))
looks like this is the relevant change:
The Iterator.collect() method in 2.7.7 returns a Seq. In 2.8, it is used to perform a conditional map using a PartialFunction. You can use input.getLines.toSeq instead.
Ah, I remember this:
EDIT: replaced with more in depth answer
The code was written against Scala
2.7.3 and 2.8 introduces some breaking changes.
Here's an update to the code that
works under Scala 2.8.0:
import scala.io._
object Sum {
def main(args: Array[String]): Unit = {
println("Enter some numbers and press ctrl-D (Unix/Mac) ctrl-Z (Windows)")
val input = Source.fromInputStream(System.in)
val lines = input.getLines.toList
println("Sum " + sum(lines))
}
def toInt(s: String): Option[Int] = {
try {
Some(Integer.parseInt(s))
} catch {
case e: NumberFormatException => None
}
}
def sum(in: Seq[String]): Int = {
val ints = in.flatMap(toInt(_))
ints.foldLeft(0)((a, b) => a + b)
}
}
Source: http://scala-programming-language.1934581.n4.nabble.com/Beginning-Scala-book-problem-td2966867.html