How to read a file in PureScript (readFile) - purescript

While testing, I’d like to read and write a file from and to disk (JSON in, JSON out). How do I do this in PureScript? In Haskell I have something like
main :: IO ()
main = do
input <- readFile "input.json"
writeFile "output.json" $ process input
All my attempts so far have failed, including trying to use readFile from import Node.FS.Sync.
BTW, if there is any other (better) way of reading in my JSON file, let me know. (I am not a JavaScript expert but I would like to port some — strict — Haskell code to JS so that it can be used elsewhere.)

Or with purescript 0.12
module Main where
import Prelude
import Effect (Effect)
import Effect.Console (log)
import Node.Encoding (Encoding(..))
import Node.FS.Sync (readTextFile)
main :: Effect Unit
main = do
log =<< readTextFile UTF8 "input.json"

Here's a full working example that reads and prints a file.
module Main where
import Prelude (Unit)
import Control.Bind ((=<<))
import Control.Monad.Eff (Eff)
import Control.Monad.Eff.Console (CONSOLE, log)
import Control.Monad.Eff.Exception (EXCEPTION)
import Node.Encoding (Encoding(..))
import Node.FS (FS)
import Node.FS.Sync (readTextFile)
main :: forall e. Eff (console :: CONSOLE, exception :: EXCEPTION, fs :: FS | e) Unit
main = do
log =<< readTextFile UTF8 "input.json"
Dependencies:
purescript-console
purescript-eff
purescript-math
purescript-node-fs-aff
purescript-control

Related

Using scala.sys.process with Future

I am trying to execute something like this
scala> import scala.sys.process._
scala> Process("cat temp.txt")!
I will be doing this say in a Play Framework REST handler. I want this to return a future object so that I can map/flatMap on it and do further processing when the shell is done executing. How do I do that?
I think all you need is this.
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.sys.process._
val fs = Future("cat temp.txt".!!) // Future[String] = Future(<not completed>)
The file contents becomes one long string but you can split() it in the map() operation.

Scala Queue import error

If I import scala.collection._ and create a queue:
import scala.collection._
var queue = new Queue[Component]();
I get the following error:
error: not found: type Queue
However, if I also add
import scala.collection.mutable.Queue
The error will disappear.
Why is this happening?
Shouldn't scala.collection._ contain scala.collection.mutable.Queue?
You have to know how the Scala collections library is structured. It splits collections based on whether they are mutable or immutable.
Queue lives in the scala.collection.mutable package and scala.collection.immutable package. You have to specify which one you want e.g.
scala> import scala.collection.mutable._
import scala.collection.mutable._
scala> var q = new Queue[Int]()
q: scala.collection.mutable.Queue[Int] = Queue()
scala> import scala.collection.immutable._
import scala.collection.immutable._
scala> var q = Queue[Int]()
q: scala.collection.immutable.Queue[Int] = Queue()
After import scala.collection._ you can use mutable.Queue; you could write Queue if there was a scala.collection.Queue (or one of scala.Queue, java.lang.Queue, and scala.Predef.Queue, since all of their members are imported by default), but there isn't.
It should be easy to see why it works this way: otherwise, the compiler (or anyone reading your code) would have no idea where they should look for the type: do you want scala.collection.Queue, scala.collection.mutable.Queue, or scala.collection.some.subpackage.from.a.library.Queue?

getOrElse method not being found in Scala Spark

Attempting to follow example in Sandy Ryza's book Advanced Analytics with Spark, coding using IntelliJ. Below I seem to have imported all the right libraries, but why is it not recognizing getOrElse?
Error:(84, 28) value getOrElse is not a member of org.apache.spark.rdd.RDD[String]
bArtistAlias.value.getOrElse(artistID, artistID)
^
Code:
import org.apache.spark.rdd.RDD
import org.apache.spark.rdd._
import org.apache.spark.rdd.PairRDDFunctions
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
import org.apache.spark.mllib.recommendation._
val trainData = rawUserArtistData.map { line =>
val Array(userID, artistID, count) = line.split(' ').map(_.toInt)
val finalArtistID = bArtistAlias.value.getOrElse(artistID, artistID)
Rating(userID, finalArtistID, count)
}.cache()
I can only make an assumption as the code listed is missing pieces, but my guess is that bArtistAlias is supposed to be a Map that SHOULD be broadcast, but isnt.
I went and found the piece of code in Sandy's book and it corroborates my guess. So, you seem to be missing this piece:
val bArtistAlias = sc.broadcast(artistAlias)
I am not even sure what you did without the code, but it looks like you broadcast an RDD[String], thus the error.....this would not even work anyway as you cannot work with another RDD inside of an RDD

Compiled Querys in Slick

I need to compile a query in Slick with Play and PostgreSQL
val bioMaterialTypes: TableQuery[Tables.BioMaterialType] = Tables.BioMaterialType
def getAllBmts() = for{ bmt <- bioMaterialTypes } yield bmt
val queryCompiled = Compiled(getAllBmts _)
but in Scala IDE I get this error in the Apply of Compiled
Multiple markers at this line
- Computation of type () => scala.slick.lifted.Query[models.Tables.BioMaterialType,models.Tables.BioMaterialTypeRow,Seq]
cannot be compiled (as type C)
- not enough arguments for method apply: (implicit compilable: scala.slick.lifted.Compilable[() =>
scala.slick.lifted.Query[models.Tables.BioMaterialType,models.Tables.BioMaterialTypeRow,Seq],C], implicit driver:
scala.slick.profile.BasicProfile)C in object Compiled. Unspecified value parameters compilable, driver.
This are my imports:
import scala.concurrent.Future
import scala.slick.jdbc.StaticQuery.staticQueryToInvoker
import scala.slick.lifted.Compiled
import scala.slick.driver.PostgresDriver
import javax.inject.Inject
import javax.inject.Singleton
import models.BioMaterialType
import models.Tables
import play.api.Application
import play.api.db.slick.Config.driver.simple.TableQuery
import play.api.db.slick.Config.driver.simple.columnExtensionMethods
import play.api.db.slick.Config.driver.simple.longColumnType
import play.api.db.slick.Config.driver.simple.queryToAppliedQueryInvoker
import play.api.db.slick.Config.driver.simple.queryToInsertInvoker
import play.api.db.slick.Config.driver.simple.stringColumnExtensionMethods
import play.api.db.slick.Config.driver.simple.stringColumnType
import play.api.db.slick.Config.driver.simple.valueToConstColumn
import play.api.db.slick.DB
import play.api.db.slick.DBAction
You can simply do
val queryCompiled = Compiled(bioMaterialTypes)

reduceByKey method not being found in IntelliJ

Here is code I'm trying out for reduceByKey :
import org.apache.spark.rdd.RDD
import org.apache.spark.SparkContext._
import org.apache.spark.SparkContext
import scala.math.random
import org.apache.spark._
import org.apache.spark.storage.StorageLevel
object MapReduce {
def main(args: Array[String]) {
val sc = new SparkContext("local[4]" , "")
val file = sc.textFile("c:/data-files/myfile.txt")
val counts = file.flatMap(line => line.split(" "))
.map(word => (word, 1))
.reduceByKey(_ + _)
}
}
Is giving compiler error : "cannot resolve symbol reduceByKey"
When I hover over implementation of reduceByKey it gives three possible implementations so it appears it is being found ?:
You need to add the following import to your file:
import org.apache.spark.SparkContext._
Spark documentation:
"In Scala, these operations are automatically available on RDDs containing Tuple2 objects (the built-in
tuples in the language, created by simply writing (a, b)), as long as you import org.apache.spark.SparkContext._ in your program to enable Spark’s implicit conversions. The key-value pair operations are available in the PairRDDFunctions class, which automatically wraps around an RDD of tuples if you import the conversions."
It seems as if the documented behavior has changed in Spark 1.4.x. To have IntelliJ recognize the implicit conversions you now have to add the following import:
import org.apache.spark.rdd.RDD._
I have noticed that at times IJ is unable to resolve methods that are imported implicitly via PairRDDFunctions https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala .
The methods implicitly imported include the reduceByKey* and reduceByKeyAndWindow* methods. I do not have a general solution at this time -except that yes you can safely ignore the intellisense errors