I'm trying to calculate the hour difference between two joda dates.
val d1 = getDateTime1()
val d2 = getDateTime2()
Here is what I did:
# attemp1
val hours = Hours.hoursBetween(d1, d2) // error - types mismatch
# attempt2
val p = Period(d1, d2, PeriodType.hours()) // error - there is such a constructor
p.getHours
So how do I do this?
The type of getDateTime1 should be org.joda.time.DateTime.
This woks fine:
val d1: org.joda.time.DateTime = DateTime.now
val d2: org.joda.time.DateTime = DateTime.nextMonth
val hours = Hours.hoursBetween(d1, d2)
// org.joda.time.Hours = PT672H
There is no factory method apply for Period, you should use new to create Period:
val p = new Period(d1, d2, PeriodType.hours())
// org.joda.time.Period = PT672H
If you are using nscala-time you could also use method to to get Interval and than convert it to Period:
d1 to d2 toPeriod PeriodType.hours
// org.joda.time.Period = PT672H
(d1 to d2).duration.getStandardHours
// Long = 672
Also try sbt clean.
Related
I'm having a very weird phenomenon occur. I'm hashing some mockup ssoids, and timing it since our application is timing critical. The first run of the function takes ~210 milliseconds, which is ridiculous and won't work. However, the 2nd, 3rd, and 4th runs take a few thousand nanoseconds each. I'm really confused, what's going on? Is there some caching I can do to make the first run just as fast?
See the code here:
object Main {
def main(args: Array[String]): Unit = {
val qq = "847bf1dc46ca22dc93259c5e857d6333"
val oo = "847bf1dc46ca22dc9eriuerie9duy45"
val xx = "909909ewufv9026854509erie9ifkf3"
val ww = "65984jg3oiqh4g3q383423932824344"
val qqq = getBytes(qq)
val ooo = getBytes(oo)
val xxx = getBytes(xx)
val www = getBytes(ww)
val t1 = System.nanoTime
val r1 = XxHash64.hashByteArray(qqq, seed = 0)
val duration = (System.nanoTime - t1)
val t2 = System.nanoTime
val r2 = XxHash64.hashByteArray(ooo, seed = 0)
val duration2 = (System.nanoTime - t2)
val t3 = System.nanoTime
val r3 = XxHash64.hashByteArray(xxx, seed = 0)
val duration3 = (System.nanoTime - t3)
val t4 = System.nanoTime
val r4 = XxHash64.hashByteArray(www, seed = 0)
val duration4 = (System.nanoTime - t4)
println(duration)
println(duration2)
println(duration3)
println(duration4)
println(r1)
println(r2)
println(r3)
println(r4)
}
(Also note I recognize this is a slipshod way of doing timings, but I've just started.)
I have my code written in Spark and Scala. Now I need to measure elapsed time of particular functions of the code.
Should I use spark.time like this? But then how can I properly assign the value of df?
val df = spark.time(myObject.retrieveData(spark, indices))
Or should I do it in this way?
def time[R](block: => R): R = {
val t0 = System.nanoTime()
val result = block // call-by-name
val t1 = System.nanoTime()
println("Elapsed time: " + (t1 - t0) + "ns")
result
}
val df = time{myObject.retrieveData(spark, indices)}
Update:
As recommended in comments, I run df.rdd.count inside myObject.retrieveData in order to materialise the DataFrame.
This question already has answers here:
Calculating the difference between two Java date instances
(45 answers)
Closed 5 years ago.
So I have a timestamp in this format:
1990-10-31 18:43:12
I want to calculate the difference between two of these timestamps.
What I have attempted:
val t1 = new java.text.SimpleDateFormat("yyyy-mm-dd HH:mm:ss")
val t2 = new java.text.SimpleDateFormat("yyyy-mm-dd HH:mm:ss")
t1.parse(timestamp1)
t2.parse(timestamp2)
But what do I do after?
Firstly, Java now has java.time (JDK 1.8 and later)
import java.time._
import java.time.format._
Then (note that formatters/parsers are now stateless, it's always safe to reuse them):
val p = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss") //note months are MM
val t1 = LocalDateTime.parse(timestamp1, p)
val t2 = LocalDateTime.parse(timestamp2, p)
Now you have two date/times, you need to convert to instants (actually, ZonedDateTime but it's enough for your purposes):
val i1 = t1.atZone(ZoneId.of("Europe/London"))
val i2 = t2.atZone(ZoneId.of("Europe/London"))
Now you have two instants, you can diff them:
import java.time.temporal._
val diff = i1.until(i2, ChronoUnit.SECONDS) //Or MILLIS, MICROS etc
Appendix
Here is the REPL session:
scala> :paste
// Entering paste mode (ctrl-D to finish)
val timestamp1 = "1990-10-31 18:43:12"
val timestamp2 = "1991-09-29 18:43:12"
import java.time._
import java.time.format._
val p = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")
val t1 = LocalDateTime.parse(timestamp1, p)
val t2 = LocalDateTime.parse(timestamp2, p)
val i1 = t1.atZone(ZoneId.of("Europe/London"))
val i2 = t2.atZone(ZoneId.of("Europe/London"))
import java.time.temporal._
val diff = i1.until(i2, ChronoUnit.SECONDS)
And here are the results:
// Exiting paste mode, now interpreting.
timestamp1: String = 1990-10-31 18:43:12
timestamp2: String = 1991-09-29 18:43:12
import java.time._
import java.time.format._
p: java.time.format.DateTimeFormatter = Value(YearOfEra,4,19,EXCEEDS_PAD)'-'Value(MonthOfYear,2)'-'Value(DayOfMonth,2)' 'Value(HourOfDay,2)':'Value(MinuteOfHour,2)':'Value(SecondOfMinute,2)
t1: java.time.LocalDateTime = 1990-10-31T18:43:12
t2: java.time.LocalDateTime = 1991-09-29T18:43:12
i1: java.time.ZonedDateTime = 1990-10-31T18:43:12Z[Europe/London]
i2: java.time.ZonedDateTime = 1991-09-29T18:43:12+01:00[Europe/London]
import java.time.temporal._
diff: Long = 28767600
I'm new to all of this; I'm currently taking CSC 101 and learning how to use Scala. This is our first assignment, and I don't know where to start. I was wondering if you could explain what each variable means and what it stands for?
I've been seeing the code below a lot, and I don't know what the percent signs mean or what they do.
val s = System.currentTimeMillis / 1000
val m = (s/60) % 60
val h = (s/60/60) % 24
Unfortunately it is not very straightforward. Here is some code for you:
import java.text.SimpleDateFormat
import java.util.Calendar
val formatter = new SimpleDateFormat("yyyy/MM/dd")
val cal = Calendar.getInstance()
val now = cal.getTime()
val thisDay = formatter.format(now)
val midnight = formatter.parse(thisDay)
// milliseconds since midnight
val msSinceMidnight = now.getTime - midnight.getTime
val secSinceMidnight = msSinceMidnight / 1000
println(secSinceMidnight)
You have to use Java APIs as shown above, or you could choose to use JodaTime library: http://www.joda.org/joda-time/.
println("What is the current number of seconds since midnight?")
val s = readInt
val m = (s/60) % 60
val h = (s/60/60) % 24
That is my current code. I just do not know how to println("") so it displays in hh:mm form. Thank you in advance for any help!
I think the answer is
"%02d:%02d".format(h, m)
based on http://www.scala-lang.org/old/node/5153
Mostly like #Floris said:
val s = System.currentTimeMillis / 1000
val m = (s/60) % 60
val h = (s/60/60) % 24
val str = "%02d:%02d".format(h, m)
// str = 22:40
Now you could print it, just like you would with regular string.
Since scala 2.10 there is string interpolation feature, which allows you to write things like:
val foo = "bar"
println(s"I'm $foo!")
// I'm bar!
But I don't think it is much readable (reminds perl):
val str = f"$h%02d:$m%02d"
// show elapsed time as hours:mins:seconds
val t1 = System.currentTimeMillis/1000
// ... run some code
val t2 = System.currentTimeMillis/1000
var elapsed_s = (t2 - t1)
// for testing invent some time: 4 hours: 20 minutes: 10 seconds
elapsed_s=4*60*60 + 20*60 + 10
val residual_s = elapsed_s % 60
val residual_m = (elapsed_s/60) % 60
val elapsed_h = (elapsed_s/60/60)
// display hours as absolute, minutes & seconds as residuals
println("Elapsed time: " + "%02d:%02d:%02d".format(elapsed_h, residual_m,
residual_s))