I'm following TDD with Idris using Idris2. I'm in chapter 6 working on the DataStore with schema. First of all for some context:
infixr 5 .+.
data Schema = SString
| SInt
| (.+.) Schema Schema
SchemaType : Schema -> Type
SchemaType SString = String
SchemaType SInt = Int
SchemaType (x .+. y) = (SchemaType x, SchemaType y)
At some point we want to format values of type SchemaType schema to display to the user. In the book this problem gets solved with a display function written like so:
display : SchemaType schema -> String
display {schema = SString} item = show item
display {schema = SInt} item = show item
display {schema = (x .+. y)} (iteml, itemr) = display iteml ++ ", " ++ display itemr
I wanna figure out if it's possible do get this working with the Show interface instead so I can just call show item.
I tried out the following:
Show (SchemaType schema) where
show {schema = SString} item = show item
show {schema = SInt} item = show item
show {schema = (x .+. y)} (x, y) = "(" ++ show x ++ ", " ++ show y ++ ")"
But it tells me that schema will get erased and therefore cannot be used.
I've tried to get idris to keep it during runtime but I'm just guessing syntax and getting errors I don't really know how to interpret.
Attempt 1:
{schema:_} -> Show (SchemaType schema) where
show {schema = SString} item = show item
show {schema = SInt} item = show item
show {schema = (x .+. y)} (x, y) = "(" ++ show x ++ ", " ++ show y ++ ")"
Throws:
Error: While processing left hand side of show. Can't match on ?postpone [no locals in scope] (Non linear pattern variable).
/home/stefan/dev/tdd-idris/SchemaDataStore.idr:27:33--27:34
23 |
24 | {schema:_} -> Show (SchemaType schema) where
25 | show {schema = SString} item = show item
26 | show {schema = SInt} item = show item
27 | show {schema = (x .+. y)} (x, y) = "(" ++ show x ++ ", " ++ show y ++ ")"
^
Attempt 2:
Show ({schema:_} -> SchemaType schema) where
show {schema = SString} item = show item
show {schema = SInt} item = show item
show {schema = (x .+. y)} (x, y) = "(" ++ show x ++ ", " ++ show y ++ ")"
Throws:
Error: While processing left hand side of show. schema is not a valid argument in show ?item.
/home/stefan/dev/tdd-idris/SchemaDataStore.idr:25:3--25:31
24 | Show ({schema:_} -> SchemaType schema) where
25 | show {schema = SString} item = show item
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Attempt 3
Show (SchemaType schema) where
show item =
case (schema, item) of
(SString, str) => show str
(SInt, int) => show int
((x .+. y), (left, right)) => "(" ++ show x ++ ", " ++ show y ++ ")"
Throws:
Error: While processing right hand side of show. Sorry, I can't find any elaboration which works. If Builtin.Pair: schema is not accessible in this context.
/home/stefan/dev/tdd-idris/SchemaDataStore.idr:26:11--26:17
24 | Show (SchemaType schema) where
25 | show item =
26 | case (schema, item) of
^^^^^^
Can someone please enlighten me? Am I trying something that's not possible, did I just get the syntax wrong?
schema is not an argument to show. I think this is what you want:
{schema : _} -> Show (SchemaType schema) where
show item =
case (schema, item) of
(SString, str) => show str
(SInt, int) => show int
((x .+. y), (left, right)) => "(" ++ show x ++ ", " ++ show y ++ ")"
However that gives another error because type classes really only work well with data and you are using it with a function. Maybe someone on the Discord (in the tag description) knows how to make it work. The Idris stack overflow is not very active.
Related
I am quite new to Scala and functional programming.
I wrote the simple codes as below, which manipulates the string by counting the word.
When the 4th comma-delimitted part is empty then, I concated only three columns, otherwise I concated all the columns including the values as code above.
But I think that it is not quite proper to the functional programming. Because I used the if statement to see the input value contains the value or not.
How to change it to the more scala-like code?
str = "aa,bb,1668268540040,34.0::aa,bb,1668268540040"
val parts = str.split("::")
for (case <- parts) {
val ret = case.map(c => if (c.value.isEmpty) {
c.columnFamily + "," + c.qualifier + "," + c.ts
} else {
c.columnFamily + "," + c.qualifier + "," + c.ts + "," + c.value
})
}
str = "aa,bb,1668268540040,34.0::aa,bb,166826434343"
val parts = str.split("::")
for (part <- parts) {
val elem = part.split(",", 4)
if (elem.length == 4) {
val Array(f, q, t, v) = elem
state.put(f + ":" + q, (v, t.toLong))
} else {
val Array(f, q, t) = elem
state.put(f + ":" + q, ("", t.toLong))
}
}
#LeviRamsey's comment tells you actually everything, but just to make your code more "scala-ish", you should avoid mutable data structures in the first place (what you're doing with state, which I think is a Map object), and use immutable data structures. About your if-else part, it's actually okay in FP, but in Scala, you can use pattern matching on a list, rather than manual length checking and using Arrays. Something like this:
parts.foldLeft(Map.empty[String, (String, Long)]) {
case (state, part) =>
part.split(",", 4).toList match {
case f :: q :: t :: v :: Nil =>
state.updated(f + ":" + q, (v, t.toLong))
case f :: q :: t :: Nil =>
state.updated(f + ":" + q, ("", t.toLong))
case _ => state // or whatever thing you want to do, in case neither 4 nor 3 elements are splitted
}
}
I have been trying to count inside a for loop, but the result just ends with a parentheses. I am just printing out the key here in map.
var count = 0
xs.foreach(x => (myMap += ((count+=1).toString+","+java.util.UUID.randomUUID.toString -> x)))
Output:
(),901e9926-be1e-4dc4-b3e3-6c3b2feea2c4
Expected output:
1,901e9926-be1e-4dc4-b3e3-6c3b2feea2c4
Within your foreach, count += 1 would be of type Unit. If I understand your question correctly, the example below (using an arbitrary xs collection) might be what you're looking for:
val xs = List("a", "b", "c", "d")
var count = 0
var myMap = Map[String, String]()
xs.foreach{ x =>
count += 1
myMap += ((count.toString + "," + java.util.UUID.randomUUID.toString) -> x)
}
myMap.keys
// res1: Iterable[String] = Set(
// 1,bd971c44-b9d0-41a0-b59f-3acbf2e0dee0, 2,5459eed9-309d-4f9c-afd7-10aced9df2a0,
// 3,5816ea42-d8ed-4beb-8b30-0376d0674700, 4,30f6f22f-1e6d-4eec-86af-5bc6734d5196
// )
In case you want a more idiomatic approach, using zip for the count and foldLeft for Map aggregation would produce similar result:
val myMap = Map[String, String]()
val resultMap = xs.zip(Stream from 1).foldLeft( myMap )(
(m, x) => m + ((x._2.toString + "," + java.util.UUID.randomUUID.toString) -> x._1)
)
What you are printing here is actually (count+=1).toString. In Scala, an assignment like this will be evaluated to Unit, which is expressed by parentheses. That's why you print () and not the value of count. If you check the count variable value afterwards you will see that it is 1 as expected.
Additionally, what you are trying to do could be expressed in a better way, e.g, you could do:
val myMap = xs.zipWithIndex.map(x => (x._2 + 1) + "," + java.util.UUID.randomUUID -> x._1).toMap
I am new to spark-scala development. I am trying to create map values in spark using scala but getting nothing printed
def createMap() : Map[String, Int] = {
var tMap:Map[String, Int] = Map()
val tDF = spark.sql("select a, b, c from temp")
for (x <- tDF) {
val k = x.getAs[Long](0) + "|" + x.getAs[Long](1)
val v = x.getAs[Int](2)
tMap += ( k -> v )
println( k -> v ) ///----------This print values
}
println("Hellllooooooooo1")
for ((k,v) <- tMap) println("key = " + k+ ", value= " + v) ////------This prints nothing
println("Hellllooooooooo2")
return tMap
}
Please suggest.
user8598832 gives how to do it properly (for some value of properly). The reason your approach doesn't work is that you're adding (k, v) to the map in an executor, but the println occurs in the driver, which generally won't see the map(s) in the executor(s) (to the extent that it might, that's just an artifact of running it in local mode not in a distributed mode).
The "right" (if collecting to driver is ever right) way to do it:
import org.apache.spark.sql.functions._
tDF.select(concat_ws("|", col("a"), col("b")), col("c")).as[(String, Int)].rdd.collectAsMap
I have a list and want to add a sequential number to duplicate elements.
val lst=List("a", "b", "c", "b", "c", "d", "b","a")
The result should be
List("a___0", "b___0", "c____0", "b___1", "c____1", "d___0", "b___2","a___1")
preserving the original order.
What I have so far:
val lb=new ListBuffer[String]()
for(i<-0 to lst.length-2) {
val lbSplit=lb.map(a=>a.split("____")(0)).distinct.toList
if(!lbSplit.contains(lst(i))){
var count=0
lb+=lst(i)+"____"+count
for(j<-i+1 to lst.length-1){
if(lst(i).equalsIgnoreCase(lst(j))) {
count+=1
lb+= lst(i)+"____"+count
}
}
}
}
which results in :
res120: scala.collection.mutable.ListBuffer[String]
= ListBuffer(a____0, a____1, b____0, b____1, b____2, c____0, c____1, d____0)
messing up the order. Also if there is a more concise way that would be great.
This should work without any mutable variables.
val lst=List("a", "b", "c", "b", "c", "d", "b","a")
lst.foldLeft((Map[String,Int]().withDefaultValue(0),List[String]())){
case ((m, l), x) => (m + (x->(m(x)+1)), x + "__" + m(x) :: l)
}._2.reverse
// res0: List[String] = List(a__0, b__0, c__0, b__1, c__1, d__0, b__2, a__1)
explanation
lst.foldLeft - Take the List of items (in this case a List[String]) and fold them (starting on the left) into a single item.
(Map[String,Int]().withDefaultValue(0),List[String]()) - In this case the new item will be a tuple of type (Map[String,Int], List[String]). We'll start the tuple with an empty Map and an empty List.
case ((m, l), x) => - Every time an element from lst is passed in to the tuple calculation we'll call that element x. We'll also receive the tuple from the previous calculation. We'll call the Map part m and we'll call the List part l.
m + (x->(m(x)+1)) - The Map part of the new tuple is created by creating/updating the count for this String (the x) and adding it to the received Map.
x + "__" + m(x) :: l - The List part of the new tuple is created by pre-pending a new String at the head.
}._2.reverse - The fold is finished. Extract the List from the tuple (the 2nd element) and reverse it to restore the original order of elements.
I think a more concise way that preserves the order would just to be to use a Map[String, Int] to keep a running total of each time you've seen a particular string. Then you can just map over lst directly and keep updating the map each time you've seen a string:
var map = Map[String, Int]()
lst.map { str =>
val count = map.getOrElse(str, 0) //get current count if in the map, otherwise zero
map += (str -> (count + 1)) //update the count
str + "__" + count
}
which will give you the following for your example:
List(a__0, b__0, c__0, b__1, c__1, d__0, b__2, a__1)
I consider that easiest to read, but if you want to avoid var then you can use foldLeft with a tuple to hold the intermediate state of the map:
lst.foldLeft((List[String](), Map[String, Int]())) { case ((list, map), str) =>
val count = map.getOrElse(str, 0)
(list :+ (str + "__" + count), map + (str -> (count + 1)))
}._1
I have the following code:
val text = "some text goes here"
val (first, rest) = text.splitAt(4)
println(first + " *" + rest)
That works fine.
However, I want to have two cases, defining "first" and "rest" outside, like this:
val text = "some text goes here"
var (first, rest) = ("", "")
if (text.contains("z")) {
(first, rest) = text.splitAt(4)
} else {
(first, rest) = text.splitAt(7)
}
println(first + " *" + rest)
But that gives me an error:
scala> | <console>:2: error: ';' expected but '=' found.
(first, rest) = text.splitAt(4)
Why is it an error to do (first, rest) = text.splitAt(4) but not to do val (first, rest) = text.splitAt(4)? And what can I do?
Edit: Can't re-assign val, changed to var. Same error
The answer by Serj gives a better way of writing this, but for an answer to your question about why your second version doesn't work, you can go to the Scala specification, which makes a distinction between variable definitions and assignments.
From "4.2 Variable Declarations and Definitions":
Variable definitions can alternatively have a pattern (§8.1) as
left-hand side. A variable definition var p = e where p is a
pattern other than a simple name or a name followed by a colon and a
type is expanded in the same way (§4.1) as a value definition val p
= e, except that the free names in p are introduced as mutable variables, not values.
From "6.15 Assignments":
The interpretation of an assignment to a simple variable x = e depends
on the definition of x. If x denotes a mutable variable, then the
assignment changes the current value of x to be the result of
evaluating the expression e.
(first, rest) here is a pattern, not a simple variable, so it works in the variable definition but not in the assignment.
First of all val is immutable, so you can't reassign it. Second, if, like all control structures in Scala, can return a value. So, you can do it like this:
val text = "some text goes here"
val (first, rest) = if (text.contains("z")) text.splitAt(4) else text.splitAt(7)
println(first + " *" + rest)
SerJ de SuDDeN answer is absolutely correct but some more details why the code you mentioned works the way it works.
val (a, b) = (1, 2)
is called an extractor of a pattern-match-expression. The value on the right side is matched to the extractor of the left side. This can be done everywhere in Scala and can have different faces. For example a pattern match on a List can look something like
scala> val head :: tail = 1 :: 2 :: 3 :: Nil
head: Int = 1
tail: List[Int] = List(2, 3)
On the right side the ::-symbol is a method of class List which prepends elements to it. On the left side the ::-symbol is an extractor of class ::, a subclass of List.
Some other places can be for-comprehensions
scala> for ((a, b) <- (1 to 3) zip (4 to 6)) println(a+b)
5
7
9
or the equivalent notation with higher-order-methods
scala> (1 to 3) zip (4 to 6) foreach { case (a, b) => println(a+b) }
5
7
9