How to save output of repeat loop in parquet file? - scala

I have a word "hi" written in loop.
implicit class Rep(n: Int) {
def times[A](f: => A) { 1 to n foreach(_ => f) }
}
// use it with
130.times { println("hi") }
How to save output?

There are some bugs in your code, here is a correct one:
implicit class Rep(n: Int) {
def times[A](f: => A): Seq[A] = { 1 to n map(_ => f) }
}
// use it with
val myHis = 130.times { "hi" } // returns Vector(hi, hi, hi, hi, hi, ...)
add = otherwise the functions return type is Unit- always add the return type explicit in such cases (: Seq[A]) and the compiler would have helped you.
use map instead of foreach as foreach returns again Unit
println("hi") returns again Unit. The last statement is returned, so it must be the value you want.
To write a file of this:
new java.io.PrintWriter("filename") { write(myHis.mkString(", ")); close }
Be aware this simple example does not handle exception properly - but I think myHis.mkString(", ") is what you are looking for.

Related

Filtering inside `for` with pattern matching

I am reading a TSV file and using using something like this:
case class Entry(entryType: Int, value: Int)
def filterEntries(): Iterator[Entry] = {
for {
line <- scala.io.Source.fromFile("filename").getLines()
} yield new Entry(line.split("\t").map(x => x.toInt))
}
Now I am both interested in filtering out entries whose entryType are set to 0 and ignoring lines with column count greater or lesser than 2 (that does not match the constructor). I was wondering if there's an idiomatic way to achieve this may be using pattern matching and unapply method in a companion object. The only thing I can think of is using .filter on the resulting iterator.
I will also accept solution not involving for loop but that returns Iterator[Entry]. They solutions must be tolerant to malformed inputs.
This is more state-of-arty:
package object liner {
implicit class R(val sc: StringContext) {
object r {
def unapplySeq(s: String): Option[Seq[String]] = sc.parts.mkString.r unapplySeq s
}
}
}
package liner {
case class Entry(entryType: Int, value: Int)
object I {
def unapply(s: String): Option[Int] = util.Try(s.toInt).toOption
}
object Test extends App {
def lines = List("1 2", "3", "", " 4 5 ", "junk", "0, 100000", "6 7 8")
def entries = lines flatMap {
case r"""\s*${I(i)}(\d+)\s+${I(j)}(\d+)\s*""" if i != 0 => Some(Entry(i, j))
case __________________________________________________ => None
}
Console println entries
}
}
Hopefully, the regex interpolator will make it into the standard distro soon, but this shows how easy it is to rig up. Also hopefully, a scanf-style interpolator will allow easy extraction with case f"$i%d".
I just started using the "elongated wildcard" in patterns to align the arrows.
There is a pupal or maybe larval regex macro:
https://github.com/som-snytt/regextractor
You can create variables in the head of the for-comprehension and then use a guard:
edit: ensure length of array
for {
line <- scala.io.Source.fromFile("filename").getLines()
arr = line.split("\t").map(x => x.toInt)
if arr.size == 2 && arr(0) != 0
} yield new Entry(arr(0), arr(1))
I have solved it using the following code:
import scala.util.{Try, Success}
val lines = List(
"1\t2",
"1\t",
"2",
"hello",
"1\t3"
)
case class Entry(val entryType: Int, val value: Int)
object Entry {
def unapply(line: String) = {
line.split("\t").map(x => Try(x.toInt)) match {
case Array(Success(entryType: Int), Success(value: Int)) => Some(Entry(entryType, value))
case _ =>
println("Malformed line: " + line)
None
}
}
}
for {
line <- lines
entryOption = Entry.unapply(line)
if entryOption.isDefined
} yield entryOption.get
The left hand side of a <- or = in a for-loop may be a fully-fledged pattern. So you may write this:
def filterEntries(): Iterator[Int] = for {
line <- scala.io.Source.fromFile("filename").getLines()
arr = line.split("\t").map(x => x.toInt)
if arr.size == 2
// now you may use pattern matching to extract the array
Array(entryType, value) = arr
if entryType == 0
} yield Entry(entryType, value)
Note that this solution will throw a NumberFormatException if a field is not convertible to an Int. If you do not want that, you'll have to encapsulate x.toInt with a Try and pattern match again.

declare variable in custom control structure in scala

I am wondering if there is a way to create a temp variable in the parameter list of a custom control structure.
Essentially, I would like create a control structure that looks something like the
for loop where I can create a variable, i, and have access to i in the loop body only:
for(i<- 1 to 100) {
//loop body can access i here
}
//i is not visible outside
I would like to do something similar in my code. For example,
customControl ( myVar <- "Task1") {
computation(myVar)
}
customControl ( myVar <- "Task2") {
computation(myVar)
}
def customControl (taskId:String) ( body: => Any) = {
Futures.future {
val result = body
result match {
case Some(x) =>
logger.info("Executed successfully")
x
case _ =>
logger.error(taskId + " failed")
None
}
}
}
Right now, I get around the problem by declaring a variable outside of the custom control structure, which doesn't look very elegant.
val myVar = "Task1"
customControl {
computation(myVar)
}
val myVar2 = "Task2"
customControl {
computation(myVar2 )
}
You could do something like this:
import scala.actors.Futures
def custom(t: String)(f: String => Any) = {
Futures.future {
val result = f(t)
result match {
case Some(x) =>
println("Executed successfully")
x
case _ =>
println(t + " failed")
None
}
}
}
And then you can get syntax like this, which isn't exactly what you asked for, but spares you declaring the variable on a separate line:
scala> custom("ss") { myvar => println("in custom " + myvar); myvar + "x" }
res7: scala.actors.Future[Any] = <function0>
in custom ss
ss failed
scala> custom("ss") { myvar => println("in custom " + myvar); Some(myvar + "x") }
in custom ss
Executed successfully
res8: scala.actors.Future[Any] = <function0>
scala>
Note that the built-in for (x <- expr) body is just syntactic sugar for
expr foreach (x => body)
Thus it might be possible to achieve what you want (using the existing for syntax) by defining a custom foreach method.
Also note that there is already a foreach method that applies to strings. You could do something like this:
case class T(t: String) {
def foreach(f: String => Unit): Unit = f(t)
}
Note: You can also change the result type of f above from Unit to Any and it will still work.
Which would enable you to do something like
for (x <- T("test"))
print(x)
This is just a trivial (and useless) example, since now for (x <- T(y)) f(x) just abbreviates (or rather "enlongishes") f(y). But of course by changing the argument of f in the above definition of foreach from String to something else and doing a corresponding translation from the string t to this type, you could achieve more useful effects.

Scala extending while loops to do-until expressions

I'm trying to do some experiment with Scala. I'd like to repeat this experiment (randomized) until the expected result comes out and get that result. If I do this with either while or do-while loop, then I need to write (suppose 'body' represents the experiment and 'cond' indicates if it's expected):
do {
val result = body
} while(!cond(result))
It does not work, however, since the last condition cannot refer to local variables from the loop body. We need to modify this control abstraction a little bit like this:
def repeat[A](body: => A)(cond: A => Boolean): A = {
val result = body
if (cond(result)) result else repeat(body)(cond)
}
It works somehow but is not perfect for me since I need to call this method by passing two parameters, e.g.:
val result = repeat(body)(a => ...)
I'm wondering whether there is a more efficient and natural way to do this so that it looks more like a built-in structure:
val result = do { body } until (a => ...)
One excellent solution for body without a return value is found in this post: How Does One Make Scala Control Abstraction in Repeat Until?, the last one-liner answer. Its body part in that answer does not return a value, so the until can be a method of the new AnyRef object, but that trick does not apply here, since we want to return A rather than AnyRef. Is there any way to achieve this? Thanks.
You're mixing programming styles and getting in trouble because of it.
Your loop is only good for heating up your processor unless you do some sort of side effect within it.
do {
val result = bodyThatPrintsOrSomething
} until (!cond(result))
So, if you're going with side-effecting code, just put the condition into a var:
var result: Whatever = _
do {
result = bodyThatPrintsOrSomething
} until (!cond(result))
or the equivalent:
var result = bodyThatPrintsOrSomething
while (!cond(result)) result = bodyThatPrintsOrSomething
Alternatively, if you take a functional approach, you're going to have to return the result of the computation anyway. Then use something like:
Iterator.continually{ bodyThatGivesAResult }.takeWhile(cond)
(there is a known annoyance of Iterator not doing a great job at taking all the good ones plus the first bad one in a list).
Or you can use your repeat method, which is tail-recursive. If you don't trust that it is, check the bytecode (with javap -c), add the #annotation.tailrec annotation so the compiler will throw an error if it is not tail-recursive, or write it as a while loop using the var method:
def repeat[A](body: => A)(cond: A => Boolean): A = {
var a = body
while (cond(a)) { a = body }
a
}
With a minor modification you can turn your current approach in a kind of mini fluent API, which results in a syntax that is close to what you want:
class run[A](body: => A) {
def until(cond: A => Boolean): A = {
val result = body
if (cond(result)) result else until(cond)
}
}
object run {
def apply[A](body: => A) = new run(body)
}
Since do is a reserved word, we have to go with run. The result would now look like this:
run {
// body with a result type A
} until (a => ...)
Edit:
I just realized that I almost reinvented what was already proposed in the linked question. One possibility to extend that approach to return a type A instead of Unit would be:
def repeat[A](body: => A) = new {
def until(condition: A => Boolean): A = {
var a = body
while (!condition(a)) { a = body }
a
}
}
Just to document a derivative of the suggestions made earlier, I went with a tail-recursive implementation of repeat { ... } until(...) that also included a limit to the number of iterations:
def repeat[A](body: => A) = new {
def until(condition: A => Boolean, attempts: Int = 10): Option[A] = {
if (attempts <= 0) None
else {
val a = body
if (condition(a)) Some(a)
else until(condition, attempts - 1)
}
}
}
This allows the loop to bail out after attempts executions of the body:
scala> import java.util.Random
import java.util.Random
scala> val r = new Random()
r: java.util.Random = java.util.Random#cb51256
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res0: Option[Int] = Some(98)
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res1: Option[Int] = Some(98)
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res2: Option[Int] = None
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res3: Option[Int] = None
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res4: Option[Int] = Some(94)

Storing an anonymous function passed as a parameter in a Map

I'm trying to implement a simple web application server as a personal project to improve my Scala, but I've hit upon a problem.
I'd like to be able to set up routes using code like the following:
def routes()
{
get("/wobble")
{
...many lines of code here...
}
get("/wibble")
{
...many lines of code here...
}
post("/wibble")
{
...many lines of code here...
}
post("/wobble")
{
...many lines of code here...
}
}
routes is called by the server when it starts and get and post are functions defined by me like this:
get(url:String)(func:()=>String)=addroute("GET",url,func)
post(url:String(func:()=>String)=addroute("POST",url,func)
addroute(method:String,url:String,f:()=>String)
{
routesmap+=(method->Map[String,()=>String](url,func))
}
Unfortunately, I've had nothing but problems with this. Could anyone tell me the correct way in Scala to add an anonymous function (as passed in as a parameter in the defined routes function above) to a Map (or any other Scala collection for that matter)?
Here is a working example:
scala> var funcs = Map[String,(Int)=>Int]()
funcs: scala.collection.immutable.Map[String,Int => Int] = Map()
scala> funcs += ("time10", i => i * 10 )
scala> funcs += ("add2", i => i + 2 )
scala> funcs("add2")(3)
res3: Int = 5
scala> funcs("time10")(10)
res4: Int = 100
You can also add a declared function:
val minus5 = (i:Int) => i - 5
funcs += ( "minus5", minus5)
Or a method:
def square(i: Int) = i*i
funcs += ("square", square)
In your case, you can have two maps, one for GET and one for POST. It should simplify the design (and at most, you will end with four maps if you include DEL and PUT).
May be, this one ? :
type Fonc = ( (=> String) => Unit)
var routesmap = Map[String,Map[String,()=>String]]()
def addRoute(method:String,url:String,f:()=>String) = {
routesmap+=(method-> (routesmap.getOrElse(method,Map[String,()=>String]()) + (url->f)))
}
def get(url:String):Fonc = (x => addRoute("GET",url,() => x))
def post(url:String):Fonc = (x => addRoute("POST",url,() => x))
def routes()
{
post("/wobble")
{
"toto"
}
get("/wibble")
{
"titi"
}
}
you can try this code :
def addRoute(method:String,url:String,f:()=>String) = {
routesmap+=(method-> (routesmap.getOrElse(method,Map[String,()=>String]()) + (url->f)))
}
def get(url:String,func:()=>String)= addRoute("GET",url,func)
def post(url:String,func:()=>String)= addRoute("POST",url,func)
def routes()
{
get("/wobble",()=>{"toto"})
get("/wibble",()=>{println("test")
"titi"})
}
and execute these commands
scala> routes
scala> routesmap.get("GET").get("/wibble")()

How could I implement an early return from outside the body of a method in Scala?

Disclaimer: Before someone says it: yes, I know it's bad style and not encouraged. I'm just doing this to play with Scala and try to learn more about how the type inference system works and how to tweak control flow. I don't intend to use this code in practice.
So: suppose I'm in a rather lengthy function, with lots of successive checks at the beginning, which, if they fail, are all supposed to cause the function to return some other value (not throw), and otherwise return the normal value. I cannot use return in the body of a Function. But can I simulate it? A bit like break is simulated in scala.util.control.Breaks?
I have come up with this:
object TestMain {
case class EarlyReturnThrowable[T](val thrower: EarlyReturn[T], val value: T) extends ControlThrowable
class EarlyReturn[T] {
def earlyReturn(value: T): Nothing = throw new EarlyReturnThrowable[T](this, value)
}
def withEarlyReturn[U](work: EarlyReturn[U] => U): U = {
val myThrower = new EarlyReturn[U]
try work(myThrower)
catch {
case EarlyReturnThrowable(`myThrower`, value) => value.asInstanceOf[U]
}
}
def main(args: Array[String]) {
val g = withEarlyReturn[Int] { block =>
if (!someCondition)
block.earlyReturn(4)
val foo = precomputeSomething
if (!someOtherCondition(foo))
block.earlyReturn(5)
val bar = normalize(foo)
if (!checkBar(bar))
block.earlyReturn(6)
val baz = bazify(bar)
if (!baz.isOK)
block.earlyReturn(7)
// now the actual, interesting part of the computation happens here
// and I would like to keep it non-nested as it is here
foo + bar + baz + 42 // just a dummy here, but in practice this is longer
}
println(g)
}
}
My checks here are obviously dummy, but the main point is that I'd like to avoid something like this, where the actually interesting code ends up being way too nested for my taste:
if (!someCondition) 4 else {
val foo = precomputeSomething
if (!someOtherCondition(foo)) 5 else {
val bar = normalize(foo)
if (!checkBar(bar)) 6 else {
val baz = bazify(bar)
if (!baz.isOK) 7 else {
// actual computation
foo + bar + baz + 42
}
}
}
}
My solution works fine here, and I can return early with 4 as return value if I want. Trouble is, I have to explicitly write the type parameter [Int] — which is a bit of a pain. Is there any way I can get around this?
It's a bit unrelated to your main question, but I think, a more effective approach (that doesn't require throwing an exception) to implement return would involve continuations:
def earlyReturn[T](ret: T): Any #cpsParam[Any, Any] = shift((k: Any => Any) => ret)
def withEarlyReturn[T](f: => T #cpsParam[T, T]): T = reset(f)
def cpsunit: Unit #cps[Any] = ()
def compute(bool: Boolean) = {
val g = withEarlyReturn {
val a = 1
if(bool) earlyReturn(4) else cpsunit
val b = 1
earlyReturn2(4, bool)
val c = 1
if(bool) earlyReturn(4) else cpsunit
a + b + c + 42
}
println(g)
}
The only problem here, is that you have to explicitly use cpsunit.
EDIT1: Yes, earlyReturn(4, cond = !checkOK) can be implemented, but it won't be that general and elegant:
def earlyReturn2[T](ret: T, cond: => Boolean): Any #cpsParam[Any, Any] =
shift((k: Any => Any) => if(cond) ret else k())
k in the snippet above represents the rest of the computation. Depending on the value of cond, we either return the value, or continue the computation.
EDIT2: Any chance we might get rid of cpsunit? The problem here is that shift inside the if statement is not allowed without else. The compiler refuses to convert Unit to Unit #cps[Unit].
I think a custom exception is the right instinct here.