Scala: insert Long into BigInt using bit shifting - scala

What I want to do to insert the current timestamp into a number of type BigInt. Everything in the BigInt should be shifted to the left to make room for the timestamp. However, leading zeros of the time stamp should be ignored.
What I'm doing at the moment:
val number: BigInt = ...
val time = System.currentTimeMillis
val usedBits = 64 - java.lang.Long.numberOfLeadingZeros(time )
val newNumber = (number << usedBits) | t
Is there a better way to do this? Especially the third line does not seem to be very elegant.

I'm not sure I followed what/why you are doing the shift with a BigInt, but you can use Strings to define BigInts and that might get you closer to what you want.
For example you had:
val number:BigInt = 999 // guessed a number
val time = System.currentTimeMillis // results in Long = 1465400383430
val usedBits = 64 - java.lang.Long.numberOfLeadingZeros(time) // results in Int = 41
val newNumber = (number << usedBits) | time // results in BigInt = 2198289632679878
You could instead change the last line to:
val newNumber2 = BigInt( number.toString + time.toString ) // BigInt = 9991465400383430
To delve int to the difference (it's only 3 bits):
newNumber.bitLength = 51
newNumber2.bitLength = 54
and the binary
newNumber.toLong.toBinaryString = " 111110011110101010100110000101010110101011111000110"
newNumber2.toLong.toBinaryString = "100011011111110010111101010001111110011011011111000110"
Until today, I would have not considered using BigInt to pack bits. But at least in this case it does the job quite well and as a bonus if you use the second method and if you know the length of your numeric strings the values can be extracted easily. Cool :-)

Related

How to generate 15 digit random number using Scala

I am new to Scala programming, I want to generate random number with 15 digits, So can you please let share some example. I have tried the below code to get the alpha number string with 10 digits.
var ranstr = s"${(Random.alphanumeric take 10).mkString}"
print("ranstr", ranstr)
You need to pay attention to the return type. You cannot have a 15-digit Int because that type is a 32-bit signed integer, meaning that it's maximum value is a little over 2B. Even getting a 10-digit number means you're at best getting a number between 1B and the maximum value of Int.
Other answers go in the detail of how to get a 15-digits number using Long. In your comment you mentioned between, but because of the limitation I mentioned before, using Ints will not allow you to go beyond the 9 digits in your example. You can, however, explicitly annotate your numeric literals with a trailing L to make them Long and achieve what you want as follows:
Random.between(100000000000000L, 1000000000000000L)
Notice that the documentation for between says that the last number is exclusive.
If you're interested in generating arbitrarily large numbers, a String might get the job done, as in the following example:
import scala.util.Random
import scala.collection.View
def nonZeroDigit: Char = Random.between(49, 58).toChar
def digit: Char = Random.between(48, 58).toChar
def randomNumber(length: Int): String = {
require(length > 0, "length must be strictly positive")
val digits = View(nonZeroDigit) ++ View.fill(length - 1)(digit)
digits.mkString
}
randomNumber(length = 1)
randomNumber(length = 10)
randomNumber(length = 15)
randomNumber(length = 40)
Notice that when converting an Int to a Char what you get is the character encoded by that number, which isn't necessarily the same as the digit represented by the Int itself. The numbers you see in the functions from the ASCII table (odds are it's good enough for what you want to do).
If you really need a numeric type, for arbitrarily large integers you will need to use BigInt. One of its constructors allows you to parse a number from a string, so you can re-use the code above as follows:
import scala.math.BigInt
BigInt(randomNumber(length = 15))
BigInt(randomNumber(length = 40))
You can play around with this code here on Scastie.
Notice that in my example, in order to keep it simple, I'm forcing the first digit of the random number to not be zero. This means that the number 0 itself will never be a possible output. If you want that to be the case if one asks for a 1-digit long number, you're advised to tailor the example to your needs.
A similar approach to that by Alin's foldLeft, based here in scanLeft, where the intermediate random digits are first collected into a Vector and then concatenated as a BigInt, while ensuring the first random digit (see initialization value in scanLeft) is greater than zero,
import scala.util.Random
import scala.math.BigInt
def randGen(n: Int): BigInt = {
val xs = (1 to n-1).scanLeft(Random.nextInt(9)+1) {
case (_,_) => Random.nextInt(10)
}
BigInt(xs.mkString)
}
To notice that Random.nextInt(9) will deliver a random value between 0 and 8, thus we add 1 to shift the possibble values from 1 to 9. Thus,
scala> (1 to 15).map(randGen(_)).foreach(println)
8
34
623
1597
28474
932674
5620336
66758916
186155185
2537294343
55233611616
338190692165
3290592067643
93234908948070
871337364826813
There a lot of ways to do this.
The most common way is to use Random.nextInt(10) to generate a digit between 0-9.
When building a number of a fixed size of digits, you have to make sure the first digit is never 0.
For that I'll use Random.nextInt(9) + 1 which guarantees generating a number between 1-9, a sequence with the other 14 generated digits, and a foldleft operation with the first digit as accumulator to generate the number:
val number =
Range(1, 15).map(_ => Random.nextInt(10)).foldLeft[Long](Random.nextInt(9) + 1) {
(acc, cur_digit) => acc * 10 + cur_digit
}
Normally for such big numbers it's better to represent them as sequence of characters instead of numbers because numbers can easily overflow. But since a 15 digit number fits in a Long and you asked for a number, I used one instead.
In scala we have scala.util.Random to get a random value (not only numeric), for a numeric value random have nextInt(n: Int) what return a random num < n. Read more about random
First example:
val random = new Random()
val digits = "0123456789".split("")
var result = ""
for (_ <- 0 until 15) {
val randomIndex = random.nextInt(digits.length)
result += digits(randomIndex)
}
println(result)
Here I create an instance of random and use a number from 0 to 9 to generate a random number of length 15
Second example:
val result2 = for (_ <- 0 until 15) yield random.nextInt(10)
println(result2.mkString)
Here I use the yield keyword to get an array of random integers from 0 to 9 and use mkString to combine the array into a string. Read more about yield

Sorting an array into a 'histogram'

I am a fairly new software Engineering student, so my knowledge is limited.
I have been given a task to sort an array, which in this case is a set of lottery numbers to give a graph like output in the console, I could do this using a whole bunch of match statements but feel there must be a better way.
this is the exact brief given:
Suppose that the following declaration defines the number of times that each of the national lottery balls (1 ..49) has been drawn over a given period.
var lottery = Array(23,16,18,19,26,13,22, /* 1 .. 7 */
20,14,22,18,21,15,17, /* 8 .. 14 */
24,15,18,20,13,14,20, /* 15 .. 21 */
18,22,20,16,19,11,20, /* 22 .. 28 */
16,28,22,20,15,17,17, /* 29 .. 35 */
21,21,19,20,14,22,25, /* 36 .. 42 */
19,17,26,18,20,23,12); /* 43 .. 49 */
write a program to print a histogram showing the information graphically using stars like so:
1 (23) | **********************
2 (16) | ************
and so on..
any hints/ advice on how to accomplish this would be appreciated as it has stumped me thus far. I am not asking for an exact solution just some guidance on what methods i could use to accomplish them as it is likely I have not come across them.
Thank you and have a good day.
Edit - the first way I completed the task.
object lottery {
def main(args: Array[String]): Unit = {
var lotteryIndex = 1
var lottery = Array(23,16,18,19,26,13,22, /* 1 .. 7 */
20,14,22,18,21,15,17, /* 8 .. 14 */
24,15,18,20,13,14,20, /* 15 .. 21 */
18,22,20,16,19,11,20, /* 22 .. 28 */
16,28,22,20,15,17,17, /* 29 .. 35 */
21,21,19,20,14,22,25, /* 36 .. 42 */
19,17,26,18,20,23,12); /* 43 .. 49 */
scala.util.Sorting.quickSort(lottery)
var i = 1
while(i < lottery.length){
if(lottery(i) != lottery(i-1)){
print("\n" + lotteryIndex + " (" + lottery(i) + ") | " + "*")
i += 1
lotteryIndex += 1
}else{
print("*")
i += 1
}
}
}
}
This is fairly straightforward. Each value in the array is the frequency count of the index, so you need to be able to create a string of asterisks ('*') whose length is the same as the frequency count. This can be achieved any number of ways, but perhaps the simplest is to define a function as follows:
def ast(fc: Int) = "*" * fc
Where fc is the frequency count and the number of asterisks required. So, for example, if fc is 5, the result of this expression will be "*****".
Next we need a function that creates a line of output, given a ball number (bn) and the corresponding frequency count (fc):
def line(bn: Int, fc: Int) = f"$bn%2d ($fc%2d) | ${ast(fc)}%s"
Let's explain what's happening here. The f prefix to the string tells Scala that it contains string formatting information and must be interpolated. (Refer to this explanation for further information.)
Whenever a $ is encountered, whatever follows is treated as an expression that needs to converted to a string in the output. If the expression isn't a simple one such as just a variable or value name (if it's a function call, for example), then the expression must be wrapped inside braces (e.g. ${...}). If a % follows the expression, then it identifies the format of the expression in a manner similar to the C language std::printf function.
So: $bn%2d converts the integer value bn into a two-digit decimal string; $fc%2d converts the integer value fc into a two-digit decimal string; and ${ast(fc)}%s calls the function ast passing fc as an argument, and then inserts the resulting string into the output. (The %s is redundant in this latter case.)
(Note: One of the comments on your original question specifies a format of %2s. While this correctly formats the output to two characters, it doesn't check that the argument type is integer, and is, therefore, not recommended.)
Now we need to iterate through the array to call this latter function for each member of the array...
Firstly, note that we need the index as well as the value (the ball number is the array index + 1, since arrays are 0-based). The first step is to convert lottery from an Array[Int] into an Array[(Int, Int)] via the zipWithIndex method. This takes each index-value pair in the lottery array, and combines them into a tuple (with the value first, then the index), placing the result in another array.
We can then map each tuple value in the result to a line of output as follows:
val lines = lottery.zipWithIndex.map {
case (fc, idx) => line(idx + 1, fc)
}
The above is a way of breaking open the tuple using a partial function. If you don't mind using tuple access members, this can also be achieved more tersely (but also more obfuscatedly) using:
val lines = lottery.zipWithIndex.map(p => line(p._2 + 1, p._1))
where p is a tuple of array index and frequency count. Choose whichever method you prefer.
Finally, we can iterate on the result to print out each line of the result:
lines.foreach(println)
This latter is a shorthand for:
lines.foreach(l => println(l))
So, putting this all together, we end up with the following:
object Lottery
extends App {
// Lottery data
val lottery = Array(23,16,18,19,26,13,22, /* 1 .. 7 */
20,14,22,18,21,15,17, /* 8 .. 14 */
24,15,18,20,13,14,20, /* 15 .. 21 */
18,22,20,16,19,11,20, /* 22 .. 28 */
16,28,22,20,15,17,17, /* 29 .. 35 */
21,21,19,20,14,22,25, /* 36 .. 42 */
19,17,26,18,20,23,12) /* 43 .. 49 */
// Get a string of asterisks of the required length.
def ast(fc: Int) = "*" * fc
// Get a line of the histogram from a ball number and frequency count.
def line(bn: Int, fc: Int) = f"$bn%2d ($fc%2d) | ${ast(fc)}%s"
// Get each line of output for the histogram.
val lines = lottery.zipWithIndex.map {
case (fc, idx) => line(idx + 1, fc)
}
// Print each line of the histogram
lines.foreach(println)
}
Note that we haven't sorted the histogram (the code as it stands matches the output specified in your brief). If you need to have balls listed in descending order of frequency count, simply sort the zipped array using the frequency count:
val lines = lottery.zipWithIndex.sortBy(-_._1).map {
case (fc, idx) => line(idx + 1, fc)
}
sortBy takes a function used as the sort value. Here, we take the frequency count and make it negative to reverse the order of the sort (if you want it sorted in ascending order of frequency count, remove the - sign).
Note that we must sort the array after zipping with the index, otherwise we'll lose the ball number association.
Some further observations:
In Scala, the use of var is heavily discouraged. Prefer val instead. (It's possible to write code that never uses var at all.)
objects (other than package objects) should typically have a capital first letter by convention.
Extending scala.App allows the object's constructor to become the main method of the program, and simplifies matters.
Unsorted output (which matches your brief):
1 (23) | ***********************
2 (16) | ****************
3 (18) | ******************
4 (19) | *******************
5 (26) | **************************
...
Sorted output:
30 (28) | ****************************
5 (26) | **************************
45 (26) | **************************
42 (25) | *************************
15 (24) | ************************
...
UPDATE
In response to your comment about needing a frequency count of the frequency counts (if I understand you correctly), here's how you would do that:
To get the frequency of each value, there are (again) many ways to do this. In this particular case, I'm going to use a foldLeft operation.
You can think of foldLeft as an accumulation operation: the first argument identifies a zero value—the initial value of the accumulator—while the second is a function which is applied to each member of the container (the Array). This latter function itself takes two arguments: the current accumulator value, and the value of the current element, and it returns a new accumulator value.
In this case, I'm going to use foldLeft to build an associative array (a SortedMap, which keeps its keys sorted in ascending order) that links each frequency count to the number of times that it occurs. (In this SortedMap, the _key_s are the frequency counts, and the value associated with each key is the frequency of that frequency count.)
Here's what that looks like (I've broken it down into a number of steps to make it more comprehensible):
// Required import.
import scala.collection.immutable.SortedMap
// Our initial, empty map, which reports a frequency of zero if a key is not present.
// Note that a `SortedMap` keeps its keys sorted.
val zero = SortedMap.empty[Int, Int].withDefaultValue(0)
// Iterate through the frequency counts (`fc`) in the `lottery` array. `fm` is the current
// status of our map.
val freq = lottery.foldLeft(zero) {(fm, fc) =>
// Determine the new count for this frequency count.
val newCount = fm(fc) + 1
// Create an association from the frequency count to this new count.
val assoc = fc -> newCount
// Add this association to the map, resulting in a new map. Any existing association
// will be replaced.
fm + assoc
}
If you've followed that, here's a terser version:
val freq = lottery.foldLeft(SortedMap.empty[Int, Int].withDefaultValue(0)) {(fm, fc) =>
fm + (fc -> (fm(fc) + 1))
}
Now all that remains is to create the histogram lines and print them:
val lines = freq.map {
case (k, v) => line(k, v)
}
lines.foreach(println)
(Note: The definition of the arguments for the line method need tweaking in light of the changes, but the behavior is identical.)
Output:
11 ( 1) | *
12 ( 1) | *
13 ( 2) | **
14 ( 3) | ***
15 ( 3) | ***
16 ( 3) | ***
17 ( 4) | ****
18 ( 5) | *****
19 ( 4) | ****
20 ( 8) | ********
21 ( 3) | ***
22 ( 5) | *****
23 ( 2) | **
24 ( 1) | *
25 ( 1) | *
26 ( 2) | **
28 ( 1) | *

Efficiently way to read binary files in scala

I'm trying to read a binary file (16 MB) in which I have only integers coded on 16 bits. So for that, I used chunks of 1 MB which gives me an array of bytes. For my own needs, I convert this byte array to a short array with the following function convert but reading this file with a buffer and convert it into a short array take me 5 seconds, is it a faster way than my solution ?
def convert(in: Array[Byte]): Array[Short] = in.grouped(2).map {
case Array(one) => (one << 8 | (0 toByte)).toShort
case Array(hi, lo) => (hi << 8 | lo).toShort
} .toArray
val startTime = System.nanoTime()
val file = new RandomAccessFile("foo","r")
val defaultBlockSize = 1 * 1024 * 1024
val byteBuffer = new Array[Byte](defaultBlockSize)
val chunkNums = (file.length / defaultBlockSize).toInt
for (i <- 1 to chunkNums) {
val seek = (i - 1) * defaultBlockSize
file.seek(seek)
file.read(byteBuffer)
val s = convert(byteBuffer)
println(byteBuffer size)
}
val stopTime = System.nanoTime()
println("Perf of = " + ((stopTime - startTime) / 1000000000.0) + " for a duration of " + duration + " s")
16 MB easily fits in memory unless you're running this on a feature phone or something. No need to chunk it and make the logic harder.
Just gulp the whole file at once with java.nio.files.Files.readAllBytes:
val buffer = java.nio.files.Files.readAllBytes(myfile.toPath)
assuming you are not stuck with Java 1.6. (If you are stuck with Java 1.6, pre-allocate your buffer size using myfile.size, and use read on a FileInputStream to get it all in one go. It's not much harder, just don't forget to close it!)
Then if you don't want to convert it yourself, you can
val bb = java.nio.ByteBuffer.wrap(buffer)
bb.order(java.nio.ByteOrder.nativeOrder)
val shorts = new Array[Short](buffer.length/2)
bb.asShortBuffer.get(shorts)
And you're done.
Note that this is all Java stuff; there's nothing Scala-specific here save the syntax.
If you're wondering why this is so much faster than your code, it's because grouped(2) boxes the bytes and places them in an array. That's three allocations for every short you want! You can do it yourself by indexing the array directly, and that will be fast, but why would you want to when ByteBuffer and friends do exactly what you need already?
If you really really care about that last (odd) byte, then you can use (buffer.length + 1)/2 for the size of shorts, and tack on a if ((buffer.length) & 1 == 1) shorts(shorts.length-1) = ((bb.get&0xFF) << 8).toShort to grab the last byte.
A couple of issues pop out:
If byteBuffer is always going to be 1024*1024 size then the case Array(one) in convert will never actually be used and therefore pattern matching is unnecessary.
Also, you can avoid the for loop with a tail recursive function. After the val byteBuffer = ... line you can replace the chunkNums and for loop with:
#scala.annotation.tailrec
def readAndConvert(b: List[Array[Short]], file : RandomAccessFile) : List[Array[Short]] = {
if(file.read(byteBuffer) < 0)
b
else {
file.skipBytes(1024*1024)
readAndConvert(b.+:(convert(byteBuffer)), file)
}
}
val sValues = readAndConvert(List.empty[Array[Short]], file)
Note: because list preppending is much faster than appending the above loop gets you the converted value in reverse order from the reading order in the file.

Different result returned using Scala Collection par in a series of runs

I have tasks that I want to execute concurrently and each task takes substantial amount of memory so I have to execute them in batches of 2 to conserve memory.
def runme(n: Int = 120) = (1 to n).grouped(2).toList.flatMap{tuple =>
tuple.par.map{x => {
println(s"Running $x")
val s = (1 to 100000).toList // intentionally to make the JVM allocate a sizeable chunk of memory
s.sum.toLong
}}
}
val result = runme()
println(result.size + " => " + result.sum)
The result I expected from the output was 120 => 84609924480 but the output was rather random. The returned collection size differed from execution to execution. Most of the time there was missing count even though all the futures were executed looking at the console. I thought flatMap waits the parallel executions in map to complete before returning the complete. What should I do to always get the right result using par? Thanks
Just for the record: changing the underlying collection in this case shouldn't change the output of your program. The problem is related to this known bug. It's fixed from 2.11.6, so if you use that (or higher) Scala version, you should not see the strange behavior.
And about the overflow, I still think that your expected value is wrong. You can check that the sum is overflowing because the list is of integers (which are 32 bit) while the total sum exceeds the integer limits. You can check it with the following snippet:
val n = 100000
val s = (1 to n).toList // your original code
val yourValue = s.sum.toLong // your original code
val correctValue = 1l * n * (n + 1) / 2 // use math formula
var bruteForceValue = 0l // in case you don't trust math :) It's Long because of 0l
for (i ← 1 to n) bruteForceValue += i // iterate through range
println(s"yourValue = $yourValue")
println(s"correctvalue = $correctValue")
println(s"bruteForceValue = $bruteForceValue")
which produces the output
yourValue = 705082704
correctvalue = 5000050000
bruteForceValue = 5000050000
Cheers!
Thanks #kaktusito.
It worked after I changed the grouped list to Vector or Seq i.e. (1 to n).grouped(2).toList.flatMap{... to (1 to n).grouped(2).toVector.flatMap{...

How to convert string to integer in JES

I am trying to do an assignment in JES a student jython program. I need to convert our student number taken as a string input variable to pass through our function i.e.
def assignment(stringID) and convert it into integers. The exact instructions are:
Step 1
Define an array called id which will store your 7 digit number as integers (the numbers you set in the array does not matter, it will be over written with your student number in the next step).
Step 2 Your student number has been passed in to your function as a String. You must separate the digits and assign them to your array id. This can do this manually line by line or using a loop. You will need to type cast each character from stringID to an integer before storing it in id.
I have tried so many different ways using the int and float functions but I am really stuck.
Thanks in advance!
>>> a = "545.2222"
>>> float(a)
545.22220000000004
>>> int(float(a))
545
I had to do some jython scripting for a websphere server. It must be a really old version of python it didn't have the ** operator or the len() function. I had to use an exception to find the end of a string.
Anyways I hope this saves someone else some time
def pow(x, y):
total = 1;
if (y > 0):
rng = y
else:
rng = -1 * y
print ("range", rng)
for itt in range (rng):
total *= x
if (y < 0):
total = 1.0 / float(total)
return total
#This will return an int if the percision restricts it from parsing decimal places
def parseNum(string, percision):
decIndex = string.index(".")
total = 0
print("decIndex: ", decIndex)
index = 0
string = string[0:decIndex] + string[decIndex + 1:]
try:
while string[index]:
if (ord(string[index]) >= ord("0") and ord(string[index]) <= ord("9")):
times = pow(10, decIndex - index - 1)
val = ord(string[index]) - ord("0")
print(times, " X ", val)
if (times < percision):
break
total += times * val
index += 1
except:
print "broke out"
return total
Warning! - make sure the string is a number. The function will not fail but you will get strange and almost assuredly, useless output.