I try to compare the tuple members (date) of an IO tuple with a normal tuple.
d1 ->(Integer, Int, Int) and d2 -> IO (Integer, Int, Int),
Is it possible to compare these two tuples?
I've tried something like that:
import Data.Time.Clock
import Data.Time.Calendar
import Data.Time.LocalTime
-- helper functions for tuples
x_fst (x,_,_) = x
x_snd (_,x,_) = x
x_trd (_,_,x) = x
getDate :: IO (Integer, Int, Int)
getDate = do
now <- getCurrentTime
tiz <- getCurrentTimeZone
let zoneNow = utcToLocalTime tiz now
let date#(year, month, day) = toGeorgian $ localDay zoneNow
return $ date -- here I will return an IO tuple -> IO (Integer, Int, Int)
compareDates :: a -> IO (Integer, Int, Int) -> IO Bool
compareDates d1 d2 = do
let year1 = x_fst d1
let year2 = x_fst d2
let month1 = x_snd d1
let month2 = x_snd d2
let day1 = x_trd d1
let day2 = x_trd d2
return $ (year1 == year2 && month1 == month2 && day1 == day2)
But I get the message that I can't compare an IO tuple with a normal tuple:
Couldn't match expected type `(Integer, Integer, Integer)`
with actual type `IO (Integer, Int, Int)`
In the second argument of `compareDates`, namely `date`
Is there a way around it? I would appreciate any help.
Thanks.
With the help of the comment / chat section I got it to work with the following code:
getDate :: IO Day
getDate = do
now <- getCurrentTime
tz <- getCurrentTimeZone
return . localDay $ utcToLocalTime tz now
main = do
d2 <- getDate
return $ fromGregorian 2019 6 15 == d2
Related
I am trying to create a function in functional programming, which recieves a normal Int value and translates it to financial chinese numbers and returns a String, for exaple: 301 = 三百零一. To begin, I have two maps, one with every digit from 0 to 9, and the other one with the exponentials, from 10, to 1000000.
val digits: Map[Int, String] = Map(0 -> "〇", 1 -> "壹", 2 -> "貳", 3 -> "參", 4 -> "肆", 5 -> "伍", 6 -> "陸", 7 -> "柒", 8 -> "捌", 9 -> "玖");
val exponent: Map[Int, String] = Map(1 -> "", 10 -> "拾", 100 -> "佰", 1000 -> "仟", 10000 -> "萬", 100000 -> "億", 1000000 -> "兆");
For the ones who don´t know, here goes a little explanation about how chinese numbers work. If you already know, don´t bother in reading this paragraph. In the chinese numbers, when you want to write a large number, for example 5000, you write the 5 and the 1000 symbols (伍仟) to intimate that you are multiplying 5 * 1000. If you have 539, it´s 5100 + 310 + 9. This would be 伍佰參拾玖. Lastly, if the number has 0´s between multiplications, it doesn´t matter how many they are, you write only one 0 between the other characters. For example: 501 = 5100 + 1. This is 伍佰〇壹. One last example for calrification: 50103 = 510000 + 1*100 + 3. This is 伍萬〇壹佰〇參.
So what I could do, is the following:
def format(unit: Int): String = {
val l = unit.toString.map(_.asDigit).toList
if(l.isEmpty) ""
else if(l.tail.isEmpty) digits(l.head)
else digits(l.head) + format(l.tail.mkString.toInt)
}
This translates the characters one by one. For example:
format(135) "壹參伍"
And I don´t know how to continue.
If I understood your problem correctly you can do something like this:
def toChineseFinancial(number: Int): String = {
val digits = number.toString.iterator.map(_.asDigit).toList
val length = digits.length
val exponents = List.tabulate(length)(n => math.pow(10, n).toInt)
val (sb, _) =
digits
.iterator
.zip(exponents.reverseIterator)
.foldLeft(new collection.mutable.StringBuilder(length * 2) -> false) {
case ((sb, flag), (digit, exp)) =>
if (digit == 0) sb -> true
else if (flag) sb.append("〇").append(digitsMap(digit)).append(exponentsMap(exp)) -> false
else sb.append(digitsMap(digit)).append(exponentsMap(exp)) -> false
}
sb.result()
}
You can see it running here.
Note: I used mutable.StringBuilder because building Strings is somewhat expensive, but if you want to avoid any kind of mutability you can easily replace it with a normal String.
I would expand the exponents Map using a simple case class for its values to cover:
numbers of magnitude 1, 10, 10^2, ..., 10^12
10's, 100's and 1000's of "萬" (10^4), "億" (10^8) and "兆" (10^12)
as shown below:
case class CNU(unit: String, factor: Int)
val exponents: Map[Long, CNU] = Map(
1L -> CNU("", 1),
10L -> CNU("拾", 1),
100L -> CNU("佰", 1),
1000L -> CNU("仟", 1),
10000L ->CNU("萬", 1),
100000L -> CNU("萬", 10),
1000000L -> CNU("萬", 100),
10000000L -> CNU("萬", 1000),
100000000L -> CNU("億", 1),
1000000000L -> CNU("億", 10),
10000000000L -> CNU("億", 100),
100000000000L -> CNU("億", 1000),
1000000000000L -> CNU("兆", 1),
10000000000000L -> CNU("兆", 10),
100000000000000L -> CNU("兆", 100),
1000000000000000L -> CNU("兆", 1000)
)
Creating the method:
val digits: Map[Int, String] = Map(
0 -> "〇", 1 -> "壹", 2 -> "貳", 3 -> "參", 4 -> "肆",
5 -> "伍", 6 -> "陸", 7 -> "柒", 8 -> "捌", 9 -> "玖"
)
def toChineseNumber(num: Long): String = {
val s = num.toString
val ds = s.map(_.asDigit).zip(s.length-1 to 0 by -1)
ds.foldRight(List.empty[String], 0){ case ((d, i), (accList, dPrev)) =>
val cnu = exponents(math.pow(10, i).toLong)
val digit =
if (d == 0) {
if (dPrev != 0 || num == 0) digits(d) else ""
}
else
digits(d)
val unit =
if (d == 0)
""
else {
if (cnu.factor == 1) cnu.unit else exponents(cnu.factor).unit
}
((digit + unit) :: accList, d)
}.
_1.mkString
}
Note that method foldRight is used to traverse and process the input number from right to left and dPrev in the tuple-accumulator is for carrying digits across iterations for handling repetitive 0's.
Testing it:
toChineseNumber(50)
// res1: String = 伍拾
toChineseNumber(30001)
// res2: String = 參萬〇壹
toChineseNumber(1023405)
// res3: String = 壹佰〇貳萬參仟肆佰〇伍
toChineseNumber(2233007788L)
// res4: String = 貳拾貳億參仟參佰〇柒仟柒佰捌拾捌
I have type 'a edge = {from: 'a; destination: 'a; weight: int}
and I want to have Printf.printf "%b\n"
( {from= 0; destination= 8; weight= 7}
< {from= 100; destination= 33; weight= -1} ) print true
so I tried this let ( < ) {weight= wa} {weight= wb} = wa < wb
but after this, the < operator only works on 'a edge, and it means that 1 < 2 will raise an error.
the reason why I want to do this is below
I write a leftist tree
type 'a leftist = Leaf | Node of 'a leftist * 'a * 'a leftist * int
let rank t = match t with Leaf -> 0 | Node (_, _, _, r) -> r
let is_empty t = rank t = 0
let rec merge t1 t2 =
match (t1, t2) with
| Leaf, _ -> t2
| _, Leaf -> t1
| Node (t1l, v1, t1r, r1), Node (t2l, v2, t2r, r2) ->
if v1 > v2 then merge t2 t1
else
let next = merge t1r t2 in
let rt1l = rank t1l and rn = rank next in
if rt1l < rn then Node (next, v1, t1l, rn + 1)
else Node (t1l, v1, next, rt1l + 1)
let insert v t = merge t (Node (Leaf, v, Leaf, 1))
let peek t = match t with Leaf -> None | Node (_, v, _, _) -> Some v
let pop t = match t with Leaf -> Leaf | Node (l, _, r, _) -> merge l r
If I cannot make < work as I expect, I must pass in a compare lambda wherever the < is used and substitute it. And I find it unattractive.
OCaml does not support adhoc polymorphism, but you can put the custom operator in a module that you can open locally only where you need it:
module Infix =
struct
let ( > ) = ...
end
...
if Infix.(rt1l < rn) then ...
That way, < will work on trees only inside Infix.( ... ) and still refer to Pervasives.(<) outside it.
My data-frame has a DateId (i.e. an integer column defining a date as a number of days since 1993-06-25). Objective is to calculate date id of the last day of month prior to each date in the column:
DateId -> _intermittent calc Date_ -> _result LastDayOfPriorMonthId_
9063 -> 2018-04-18 -> 9045 (i.e. 2018-03-31)
8771 -> 2017-06-30 -> 8741 (i.e. 2017-05-31)
9175 -> 2018-08-08 -> 9167 (i.e. 2018-07-31)
Solution would be really easy, but I'm running into issues with type conversion:
val a = Seq(9063, 8771, 9175).toDF("DateId")
val timeStart = to_date(lit("1993-06-25"))
val dateIdAdd : (Column) => Column = x => {x - date_add(timeStart, x).DATE_OF_MONTH}
The function compilation is failing with following error:
notebook:2: error: type mismatch;
found : org.apache.spark.sql.Column
required: Int
x - date_add(timeStart, x).DATE_OF_MONTH
Expressions like .cast(IntegerType) do not change the outcome (x is still a spark Column type and .cast(Int) is not applicable.
Please note: similar problem was addressed in this SO question, but the same approach is failing when the timeStart constant is applied here. Also using function would be preferred over expression, because the same calculation is used multiple columns with real data.
Can you translate from Java? Sorry, I don’t code Scala (yet).
private static final LocalDate baseDate = LocalDate.of(1993, Month.JUNE, 25);
public static long dateIdAdd(long dateId) {
LocalDate date = baseDate.plusDays(dateId);
LocalDate lastOfPrevMonth = YearMonth.from(date).minusMonths(1).atEndOfMonth();
return ChronoUnit.DAYS.between(baseDate, lastOfPrevMonth);
}
Edit: according to you (Dan, the asker), the Scala version is:
val baseDate = LocalDate.of(1993, Month.JUNE, 25)
val lastDayIdOfPriorMonth = udf((dateId : Long) => {
val date = baseDate.plusDays(dateId)
val lastOfPrevMonth = YearMonth.from(date).minusMonths(1).atEndOfMonth()
ChronoUnit.DAYS.between(baseDate, lastOfPrevMonth)
})
Let’s try it with your example dates (Java again):
System.out.println("9063 -> " + dateIdAdd(9063));
System.out.println("8771 -> " + dateIdAdd(8771));
System.out.println("9175 -> " + dateIdAdd(9175));
This prints:
9063 -> 9045
8771 -> 8741
9175 -> 9167
In your question you gave 9176 as desired result in the last case, but I believe that was a typo?
And please enjoy how clear and self-explanatory the code is.
After testing many options with Scala conversion function, hack based on UDF with Java string and SimpleDateFormat the only thing I could figure out:
val dateIdAdd = udf((dateId : Long) => {
val d = new SimpleDateFormat("yyyy-MM-dd")
val ts = d.parse("1993-06-25")
val tc = d.format(new Date(ts.getTime() + (24 * 3600 * 1000 * dateId)))
dateId - Integer.parseInt(tc.substring(tc.length()-2))
})
After adding another support function for validation and a simple select:
val dateIdToDate = udf((dateId : Long) => {
val d = new SimpleDateFormat("yyyy-MM-dd")
val ts = d.parse("1993-06-25")
d.format(new Date(ts.getTime() + (24 * 3600 * 1000 * dateId)))
})
val aa = a.select($"*"
, dateIdToDate($"DateId") as "CalcDateFromId"
, dateIdAdd($"DateId") as "CalcLastDayOfMonthId")
display(aa)
Expected results are generated (but I doubt this is the most efficient way available):
DateId CalcDateFromId CalcLastDayOfMonthId
9063 4/18/2018 9045
8771 6/30/2017 8741
9175 8/8/2018 9167
Is there anyway to write the following function more elegantly?
I can see some patterns but I'm not sure how to abstract them or how to find a simpler way to write the function.
type HasRemainder = Boolean
tomorrow :: Date -> Date
tomorrow date = unsafePartial $ canonicalDate y (fst m) (fst d)
where d :: Tuple Day HasRemainder
d = case toEnum $ 1 + fromEnum (day date) of
Just v -> Tuple v false
Nothing -> Tuple (unsafePartial $ fromJust $ toEnum 1) true
m :: Tuple Month HasRemainder
m = if snd d then
case toEnum $ 1 + fromEnum (month date) of
Just v -> Tuple v false
Nothing -> Tuple (unsafePartial $ fromJust $ toEnum 1) true
else Tuple (month date) false
y :: Year
y = if snd m then
case toEnum $ 1 + fromEnum (year date) of
Just v -> v
-- use 2018 arbitrarly if the conversion from int to Date fails
Nothing -> unsafePartial $ fromJust $ toEnum 2018
else (year date)
I'd do something like this:
import Data.DateTime as DT
import Data.Maybe (maybe)
import Data.Time.Duration (Days(..))
tomorrow :: DT.Date -> DT.Date
tomorrow dt = maybe dt DT.date $ DT.adjust (Days 1.0) (DT.DateTime dt bottom)
Although it will return the input date in the unlikely event that the given date is top (which is 31st December 275759 if I remember correctly).
There is an adjust function for Time and DateTime so it's just an oversight that Date is missing one.
I would try something along those lines
getDatePart datepart defaultval1 defaultval2 =
case toEnum $ defaultval1 + fromEnum datepart of
Just v -> Tuple v false
Nothing -> Tuple (unsafePartial $ fromJust $ toEnum defaultval2) true
getDatePartDefault d datepart defaultval1 defaultval2 =
if snd d then
getDatePart datepart defaultval1 defaultval2
else Tuple datepart false
tomorrow :: Date -> Date
tomorrow date = unsafePartial $ canonicalDate (fst y) (fst m) (fst d)
where d :: Tuple Day HasRemainder
d = getDatePart (day date) 1 1
m :: Tuple Month HasRemainder
m = getDatePartDefault d (month date) 1 1
y :: Tuple Year HasRemainder
y = getDatePartDefault d (year date) 1 2018
Take care: this is not tested
Using succ from Enum:
import Data.Date (Date)
import Data.Enum (succ)
import Data.Maybe (Maybe)
today :: Date
today = ...
tomorrow :: Maybe Date
tomorrow = succ today
I have a cartesian RDD which allows me to filter a RDD on a certain time range, but I need to get the minimum value of the RDD so I can calculate the delta time of each record to the entry that occurred first.
I have a case class that is made up like the below:
case class auction(id: String, prodID: String, timestamp: Long)
and I put together two RDDs, one that contains the auction of note, the other contains the auctions that occured in that time period as below:
val specificmessages = allauctions.cartesian(winningauction)
.filter( (x, y) => x.timestamp > y.timestamp - 10 &&
x.timestamp < y.timestamp + 10 &&
x.productID == y.productID )
I would like to, in the specificmessages function, be able to add a field which will contain the delta between each record and the auction timestamp that has the minimum value.
You can use DataFrames like this:
import org.apache.spark.sql.{functions => f}
import org.apache.spark.sql.expressions.Window
// Convert RDDs to DFs
val allDF = allauctions.toDF
val winDF = winningauction.toDF("winId", "winProdId", "winTimestamp")
// Prepare join conditions
val prodCond = $"prodID" === $"winProdID"
val tsCond = f.abs($"timestamp" - $"winTimestamp") < 10
// Create window
val w = Window
.partitionBy($"id", $"prodID", $"timestamp")
.orderBy($"winTimestamp")
val joined = allDF
.join(winDF, prodCond && tsCond)
.select($"*", first($"winTimestamp").over(w).alias("mintimestamp")
Using plain RDDs
// Create PairRDDs
def allPairs = allauctions.map(a => (a.prodID, a))
def winPairs = winauctions.map(a => (a.prodID, a))
allPairs
.join(winPairs) // Join by prodId -> RDD[(prodID, (auction, auction))]
// Filter timestamp
.filter{case (_, (x, y)) => (x.timestamp - y.timestamp).abs < 10} //
.values // Drop key -> RDD[(auction, auction)]
.groupByKey // Group by allAuctions -> RDD[(auction, Seq[auction])]
.flatMap{ case (k, vals) => {
val minTs = vals.map(_.timestamp).min // Find min ts from winauction
vals.map(v => (k, v, minTs))
}} // -> RDD[(auction, auction, ts)]