Synchronize on a var in Scala - scala

The problem is to search among a few pages of text for an exclamation point concurrently and as soon as any of the threads finds it, all the other threads should stop searching.
Code:
object AntiVolatile {
val pages = for (i <- 1 to 15) yield new Page("!Na" * rand.nextInt(1000) + " Batman!", -1)
var found = Some(false)
def run(): Unit = {
for (p <- pages) yield thread {
var i = 0
var foundInThread = found.get
while (i < p.txt.length && !foundInThread)
if (p.txt(i) == '!') {
found.synchronized {
found match {
case Some(true) => foundInThread = true
case Some(false) => {
p.position = i
found = Some(true)
Thread.sleep(1)
}
case _ =>
}
}
} else i += 1
// if still not found, wait for another thread to find it.
def wait(): Unit = {
found match {
case Some(false) => wait()
case _ =>
}
}
wait()
log(s"results: ${pages.map(_.position)}")
}
}
}
It seems to work ok:
Thread-29: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-27: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-28: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-26: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-30: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-31: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-32: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-25: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-33: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-34: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-39: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-38: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-37: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-36: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
Thread-35: results: Vector(0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
But then I realized that found is not a constant instance because it's reassign to a new Option object later. (Why does the code actually work?)
So I came up with a fix:
object AntiVolatile {
case class Found(var isFound: Boolean)
val pages = for (i <- 1 to 15) yield new Page("!Na" * rand.nextInt(1000) + " Batman!", -1)
val found = Found(false)
def run(): Unit = {
for (p <- pages) yield thread {
var i = 0
var foundInThread = found.isFound
while (i < p.txt.length && !foundInThread)
if (p.txt(i) == '!') {
found.synchronized {
found match {
case Found(true) => foundInThread = true
case Found(false) => {
p.position = i
found.isFound = true
Thread.sleep(1)
}
case _ =>
}
}
} else i += 1
// if still not found, wait for another thread to find it.
def wait(): Unit = {
found match {
case Found(false) => wait()
case _ =>
}
}
wait()
log(s"results: ${pages.map(_.position)}")
}
}
}
These two versions seem to behave the same, why? I would expect some bug cropping up in the first version.
Link to github repo: https://github.com/kindlychung/learnConcurrentScala/blob/master/src/main/scala/org/learningconcurrency/ch2/Ch2.scala

It is not entirely clear if you are interested in learning concurrency or if you are solving an actual problem. With that being said, I am going to assume that you are attempting to solve the problem.
Why not use Futures?
import java.util.concurrent.TimeUnit
import scala.concurrent.duration.Duration
import scala.concurrent.{ExecutionContext, Await, Future}
import scala.util.Random
import ExecutionContext.Implicits.global
object Main extends App {
case class Page(number: Int, text: String)
val pages = for (i <- 1 to 15) yield Page(i, "!Na" * Random.nextInt(1000) + " Batman! ")
val searchFutures = pages.map { p => Future {
val position = p.text.indexOf("!")
s"Exclamation mark found on page ${p.number} at position: $position"
}}
val firstCompleted = Future.firstCompletedOf(searchFutures)
val result = Await.result(firstCompleted, Duration(5, TimeUnit.SECONDS))
println(result)
}

Related

count number of groups of certain element

I have a list, I need to count the number of groups of certain elements(1 or 0).
(-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, 1, 1, 1, 0, -1, -1, -1, -1, -1, -1, -1, -1, -1)
val count1Grp = daysSequence.foldLeft(List[Int]()){(acc, elem) =>
if(acc.isEmpty) elem :: acc else if(acc.head == elem) acc else elem :: acc
}
.count(_ == 1)
val count0Grp = daysSequence.foldLeft(List[Int]()){(acc, elem) =>
if(acc.isEmpty) elem :: acc else if(acc.head == elem) acc else elem :: acc
}
.count(_ == 0)
I fold similar neighbour elements and then do a count of what is left. And get
count1Grp = 1
count0Grp = 1
Another example. Both rows same result.
(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
(-1, -1, -1, -1, -1, -1, -1, -1, -1, 1, 0, 0, 0, 0, 0, 0, -1, 0, 0, 0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
count1Grp = 1
count0Grp = 2
But how should I do it if I needed 2 or more similar elements together to quantify as a group? In the first case 0 wouldn't fit the criteria and should result in
count1Grp = 1
count0Grp = 0
Why not get a Map with a count of every group in the collection? Then only one traversal is needed.
def groupCount[A](xs : Seq[A]
,grpSize : Int = 1
,acc : Map[A,Int] = Map[A,Int]()
): Map[A,Int] = xs match {
case Seq() => acc
case hd +: _ =>
val (grp,rest) = xs.span(_ == hd)
if (grp.size < grpSize)
groupCount(rest, grpSize, acc)
else
groupCount(rest, grpSize, acc + (hd -> (acc.getOrElse(hd,0)+1)))
}
Here's a Scastie session with a few usage examples.

replace null value in the column of a dataframe with the value in other dataframe wrt to id

I have two dataframe
df1 :
+---------------+-------------------+-----+------------------------+------------------------+---------+
|id |dt |speed|stats |lag_stat |lag_speed|
+---------------+-------------------+-----+------------------------+------------------------+---------+
|358899055773504|2018-07-31 18:38:36|0 |[9, -1, -1, 13, 0, 1, 0]|null |null |
|358899055773504|2018-07-31 18:58:34|0 |[9, 0, -1, 22, 0, 1, 0] |[9, -1, -1, 13, 0, 1, 0]|0 |
|358899055773505|2018-07-31 18:54:23|4 |[9, 0, 0, 22, 1, 1, 1] |null |null |
+---------------+-------------------+-----+------------------------+------------------------+---------+
df2 :
+---------------+-------------------+-----+------------------------+
|id |dt |speed|stats |
+---------------+-------------------+-----+------------------------+
|358899055773504|2018-07-31 18:38:34|0 |[9, -1, -1, 13, 0, 1, 0]|
|358899055773505|2018-07-31 18:48:23|4 |[8, -1, 0, 22, 1, 1, 1] |
+---------------+-------------------+-----+------------------------+
I want to replace the null value in column lag_stat,speed in df1 with the value of stat and speed from dataframe df2 wrt to the same id.
Desired output looks like this:
+---------------+-------------------+-----+--------------------+--------------------+---------+
| id| dt|speed| stats| lag_stat|lag_speed|
+---------------+-------------------+-----+--------------------+--------------------+---------+
|358899055773504|2018-07-31 18:38:36| 0|[9, -1, -1, 13, 0, 1,0]|[9, -1, -1, 13, 0, 1, 0]| 0|
|358899055773504|2018-07-31 18:58:34| 0|[9, 0, -1, 22, 0, 1, 0]|[9, -1, -1, 13, 0, 1, 0]| 0|
|358899055773505|2018-07-31 18:54:23| 4|[9, 0, 0, 22, 1, 1, 1]|[8, -1, 0, 22, 1, 1, 1] | 4 |
+---------------+-------------------+-----+--------------------+--------------------+---------+
One possible way could be join the DFs and then apply some when functions on that columns.
For example, this:
val output = df1.join(df2, df1.col("id")===df2.col("id"))
.select(df1.col("id"),
df1.col("dt"),
df1.col("speed"),
df1.col("stats"),
when(df1.col("lag_stat").isNull,df2.col("stats")).otherwise(df1.col("lag_stat")).alias("lag_stats"),
when(df1.col("lag_speed").isNull,df2.col("speed")).otherwise(df1.col("lag_speed")).alias("lag_speed")
)
will give you the expected output:
+---------------+------------------+-----+------------------+------------------+---------+
| id| dt|speed| stats| lag_stats|lag_speed|
+---------------+------------------+-----+------------------+------------------+---------+
|358899055773504|2018-07-3118:38:36| 0|[9,-1,-1,13,0,1,0]|[9,-1,-1,13,0,1,0]| 0|
|358899055773504|2018-07-3118:58:34| 0| [9,0,-1,22,0,1,0]|[9,-1,-1,13,0,1,0]| 0|
|358899055773505|2018-07-3118:54:23| 4| [9,0,0,22,1,1,1]| [8,-1,0,22,1,1,1]| 4|
+---------------+------------------+-----+------------------+------------------+---------+

Remove zero rows from a list of list in Scala

I have a list of list in Scala such as:
val lst = List(List(60, 0, 1, 2, 3, 28, 0, 0, 0, 0), List(0, 0, 0, 0, 0, 0, 0, 0, 0, 0), List(47, 0, 1, 1, 2, 28, 0, 0, 0, 0))
and I want to remove all zero rows and the result should be like:
List(List(60, 0, 1, 2, 3, 28, 0, 0, 0, 0), List(47, 0, 1, 1, 2, 28, 0, 0, 0, 0))
Does Scala list have any built-in method to remove these rows?
You can use filter to keep only items (lists) matching a predicate; The predicate can use exists to check for non-zero elements:
lst.filter(_.exists(_ != 0))
#Tzach Zohar answer is perfectly fine but here is another way to approach it.
scala> lst.filterNot(xs => xs.forall(_ == 0))
res0: List[List[Int]] = List(
List(60, 0, 1, 2, 3, 28, 0, 0, 0, 0),
List(47, 0, 1, 1, 2, 28, 0, 0, 0, 0)
)

How to carry out a counter number in parallel_for_each

1)Why the variable length in my sample code are not 62 after calculate ? It's
seems every time the condition is satisfied,but enter the condition,
the number "length not be added every single time.
2)If I don't use concurrency::array length(1, 1, &V[0]); to save counter
but using tile_static int, the length is also wrong.
//if my 8 x 8 local data are:
//cache[TS][TS]
//{
// -69, 0, 0, 1, 0, 0, 0, 0,
// 0, 0, 0, 0, 0, 0, 0, 0,
// 0, 0, 0, 0, 0, 0, 0, 0,
// 0, 0, 0, 0, 0, 0, 0, 0,
// 0, 0, 0, 0, 0, 0, 0, 0,
// 0, 0, 0, 0, 0, 0, 0, 0,
// 0, 0, 0, 0, 0, 0, 0, 0,
// 0, 0, 0, 0, 0, 0, 0, 0
//}
My sample code:
void sample()
{
std::vector<int> V;
V.push_back(0);
concurrency::array<int, 2> length(1, 1, &V[0]);
const int TS = 8;
concurrency::parallel_for_each(data.extent.tile<TS, TS>(), [=, &length](tiled_index<TS, TS> index) restrict(amp)
{
const int row = index.local[0];
const int col = index.local[1];
//tile_static int length; ---------2)
tile_static int cache[TS][TS];
cache[row][col] = data[index.global];
index.barrier.wait();
if (cache[row][col] == 0)
{
//length++; -------------------2)
length[0][0] = length[0][0] + 1;
}
});
}
std::vector<int> V;
for (int i = 0; i < 256; i++)
V.push_back(0);
concurrency::array_view<int, 1> AC_count(256, &V[0]);
const int TS = 8;
concurrency::parallel_for_each(data.extent.tile<TS, TS>(), [=, &length](tiled_index<TS, TS> index) restrict(amp)
{
const int row = index.local[0];
const int col = index.local[1];
tile_static int cache[TS][TS];
cache[row][col] = data[index.global];
index.barrier.wait();
if (cache[row][col] == 0)
{
atomic_fetch_add(&AC_count[0], 1);
}
});
}

How do I accumulate results without using a mutable ArrayBuffer?

The code at the end of this question replaces the zeros with possible numbers ranging from 1 to 9 once and non-repeating. For a given sequence of numbers, List(0, 0, 1, 5, 0, 0, 8, 0, 0), it will returns the following result. There are 720 permutations in total.
List(2, 3, 1, 5, 4, 6, 8, 7, 9)
List(2, 3, 1, 5, 4, 6, 8, 9, 7)
List(2, 3, 1, 5, 4, 7, 8, 6, 9)
List(2, 3, 1, 5, 4, 7, 8, 9, 6)
List(2, 3, 1, 5, 4, 9, 8, 6, 7)
List(2, 3, 1, 5, 4, 9, 8, 7, 6)
List(2, 3, 1, 5, 6, 4, 8, 7, 9)
...
My question is how do I convert my code to NOT using ArrayBuffer(coll) as my temporary storage and the final result is returned from the function(search0) instead?
Thanks
/lim/
import collection.mutable.ArrayBuffer
object ScratchPad extends App {
def search(l : List[Int]) : ArrayBuffer[List[Int]] = {
def search0(la : List[Int], pos : Int, occur : List[Int], coll : ArrayBuffer[List[Int]]) : Unit = {
if (pos == l.length) {println(la); coll += la }
val bal = (1 to 9) diff occur
if (!bal.isEmpty) {
la(pos) match {
case 0 => bal map { x => search0(la.updated(pos, x), pos + 1, x :: occur, coll)}
case n => if (occur contains n) Nil else search0(la, pos + 1, n :: occur, coll)
}
}
}
val coll = ArrayBuffer[List[Int]]()
search0(l, 0, Nil, coll)
coll
}
println(search(List(0, 0, 1, 5, 0, 0, 8, 0, 0)).size)
}
Here is a shorter solution using immutable collection:
scala> def search(xs: Seq[Int])(implicit ys: Seq[Int] = (1 to 9).diff(xs)): Seq[Seq[Int]] = ys match {
| case Seq() => Seq(xs)
| case _ => ys.flatten(y => search(xs.updated(xs.indexOf(0), y))(ys.diff(Seq(y))))
| }
search: (xs: Seq[Int])(implicit ys: Seq[Int])Seq[Seq[Int]]
scala> search(List(0, 0, 1, 5, 0, 0, 8, 0, 0)).size
res0: Int = 720
scala> search(List(0, 0, 1, 5, 0, 0, 8, 0, 0)) take 10 foreach println
List(2, 3, 1, 5, 4, 6, 8, 7, 9)
List(2, 3, 1, 5, 4, 6, 8, 9, 7)
List(2, 3, 1, 5, 4, 7, 8, 6, 9)
List(2, 3, 1, 5, 4, 7, 8, 9, 6)
List(2, 3, 1, 5, 4, 9, 8, 6, 7)
List(2, 3, 1, 5, 4, 9, 8, 7, 6)
List(2, 3, 1, 5, 6, 4, 8, 7, 9)
List(2, 3, 1, 5, 6, 4, 8, 9, 7)
List(2, 3, 1, 5, 6, 7, 8, 4, 9)
List(2, 3, 1, 5, 6, 7, 8, 9, 4)
An even more shorter solution:
scala> :paste
// Entering paste mode (ctrl-D to finish)
def search(xs: Seq[Int], ys: Seq[Int] = 1 to 9): Seq[Seq[Int]] = ys.diff(xs) match {
case Seq() => Seq(xs)
case zs => zs.flatten(z => search(xs.updated(xs.indexOf(0),z),zs))
}
// Exiting paste mode, now interpreting.
search: (xs: Seq[Int], ys: Seq[Int])Seq[Seq[Int]]
scala> search(List(0, 0, 1, 5, 0, 0, 8, 0, 0)).size
res1: Int = 720
scala> search(List(0, 0, 1, 5, 0, 0, 8, 0, 0)) take 10 foreach println
List(2, 3, 1, 5, 4, 6, 8, 7, 9)
List(2, 3, 1, 5, 4, 6, 8, 9, 7)
List(2, 3, 1, 5, 4, 7, 8, 6, 9)
List(2, 3, 1, 5, 4, 7, 8, 9, 6)
List(2, 3, 1, 5, 4, 9, 8, 6, 7)
List(2, 3, 1, 5, 4, 9, 8, 7, 6)
List(2, 3, 1, 5, 6, 4, 8, 7, 9)
List(2, 3, 1, 5, 6, 4, 8, 9, 7)
List(2, 3, 1, 5, 6, 7, 8, 4, 9)
List(2, 3, 1, 5, 6, 7, 8, 9, 4)
Naive refactoring of your code using only immutable data structures:
object ScratchPad extends App {
def search(l: List[Int]): List[List[Int]] = {
def search0(la: List[Int], pos: Int, occur: List[Int]): List[List[Int]] =
if (pos == l.length)
List(la)
else {
val bal = (1 to 9) diff occur
if (pos < l.length && !bal.isEmpty)
la(pos) match {
case 0 => bal.toList flatMap {x => search0(la.updated(pos, x), pos + 1, x :: occur)}
case n => if (occur contains n) List.empty[List[Int]] else search0(la, pos + 1, n :: occur)
}
else
List.empty[List[Int]]
}
search0(l, 0, Nil)
}
val result = search(List(0, 0, 1, 5, 0, 0, 8, 0, 0))
result foreach println
println(result.size)
}