I have an initial date as a String which I need to convert to date with a specific format. I tried to defined date format in a string and parse it, then I formatted it to the desired format. The problem is that I need a date as a final result.
Here is the code I used:
def parseDateToOriginal(date: String): String = {
val initialDateFormat = new SimpleDateFormat("EEE MMM dd hh:mm:ss zzz yyyy")
val finalDateFormat = new SimpleDateFormat("yyyy-mm-dd")
val result = finalDateFormat.format(initialDateFormat.parse(date))
result
}
So I need Date as the return type for this method. I tried to parse the result string to get a proper date but for some reason, the result defaults back to the original date format. How can I fix this problem?
Here is how I tried to parse it again:
val parsedDate = new SimpleDateFormat("yyyy-mm-dd").parse(parseDateToOriginal(date))
The result is of the pattern "EEE MMM dd hh:mm:ss zzz yyyy"
First, SimpleDate is old and outdated. The current java.time library is recommended.
Next, if you need to return a Date then parse the input and return the Date. You need to format a Date only when you present it, i.e. change it to a String.
import java.time.LocalDate
import java.time.format.DateTimeFormatter
def parseToDate(date: String): LocalDate =
LocalDate.parse(date
,DateTimeFormatter.ofPattern("EEE MMM dd hh:mm:ss zzz yyyy"))
Try
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter
def parseDateToOriginal(date: String): String = {
LocalDateTime
.parse(date, DateTimeFormatter.ofPattern("EEE MMM d HH:mm:ss zzz yyyy"))
.format(DateTimeFormatter.ofPattern("yyyy-MM-dd"))
}
which outputs
parseDateToOriginal("Thu Jun 18 20:56:02 EDT 2009") // res2: String = 2009-06-18
Note you have a bug in the format of finalDateFormat
val finalDateFormat = new SimpleDateFormat("yyyy-mm-dd")
You are using lowercase mm in the month positions, but should be upper case MM. Lowercase mm represents minutes, so it would erroneously result in res2: String = 2009-56-18 as outputs.
Related
I am new to Pyspark
I am trying to convert a string with value Jun 22 2021 1:04PM to a timestamp using the below code block but its making the value as null, where as its showing the datatype is timestamp
df = df.withColumn("date", F.from_unixtime(F.unix_timestamp("date","MMM d, yyyy hh:mm:ss a"),'yyyy-MM-dd').cast('timestamp'))
Your date is of the format MMM d yyyy hh:mmaa
To convert a string like above format. Do like below
from pyspark.sql import functions as f
df.withColumn("date_2", f.from_unixtime(f.unix_timestamp("date", 'MMM d yyyy hh:mmaa'),'MM-dd-yyyy HH:mm:ss')).show()
try this one:
df=df.withColumn("date", from_unixtime(unix_timestamp(col("date"), "MMM d, yyyy hh:mm:ss a"),"yyyy-MM-dd")).show(false)
I am reading a CSV that contains two types of date:
dd-MMM-yyyy hh:mm:ss -> 13-Dec-2019 17:10:00
dd/MM/yyyy hh:mm -> 11/02/2020 17:33
I am trying to transform all dates of the first type into the second type but I can't find a good solution. I am trying this:
val pr_date = readeve.withColumn("Date", when(to_date(col("Date"),"dd-MMM-yyyy hh:mm:ss").isNotNull,
to_date(col("Date"),"dd/MM/yyyy hh:mm")))
pr_date.show(25)
And I get the entire Date column as null values:
I am trying with this function:
def to_date_(col: Column,
formats: Seq[String] = Seq("dd-MMM-yyyy hh:mm:ss", "dd/MM/yyyy hh:mm")) = {
coalesce(formats.map(f => to_date(col, f)): _*)
}
val p2 = readeve.withColumn("Date",to_date_(readeve.col(("Date")))).show(125)
And in the first type of date i get nulls too:
What am I doing wrong? (new with Scala Spark)
Scala version: 2.11.7
Spark version: 2.4.3
Try code below? Note that 17 is HH, not hh. Also try to_timestamp instead of to_date because you want to keep the time.
val pr_date = readeve.withColumn(
"Date",
coalesce(
date_format(to_timestamp(col("Date"),"dd-MMM-yyyy HH:mm:ss"),"dd/MM/yyyy HH:mm"),
date_format(to_timestamp(col("Date"),"dd/MM/yyyy HH:mm"),"dd/MM/yyyy HH:mm")
)
)
I have a date of string type "2020-08-10". How to convert my string date to this format Monday 08 2020 in Kotlin?
Code:
var parsedDate = LocalDate.parse("2020-08-10", DateTimeFormatter.ofPattern("yyyy-MM-dd"))
println("2020-08-10 : "+parsedDate.dayOfWeek.toString()+" "+parsedDate.monthValue+" "+parsedDate.year)
Output:
2020-08-10 : MONDAY 8 2020
For API 26 Below:
val parser = SimpleDateFormat("yyyy-MM-dd")
val formatter = SimpleDateFormat("EEEE MM yyyy")
val formattedDate = formatter.format(parser.parse("2020-08-10"))
println("2020-08-10 : "+formattedDate)
Output:
2020-08-10 : MONDAY 8 2020
The EEEE prints the name of day
import java.time.LocalDate
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter
val str = "2020-08-10"
val formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd")
val dateTime = LocalDate.parse(str, formatter)
println(dateTime.format(DateTimeFormatter.ofPattern("EEEE MM yyyy ")))
Output
Monday 08 2020
It's an alternative solution to your question.
So i have dd/mm/yyyy date format and i want to convert it into date dd MMM, yyyy format.
This is what i have try:
val s: String = "27/04/2014"
val simpleDateFormat: SimpleDateFormat = new SimpleDateFormat("dd/mm/yyyy")
val date2 = simpleDateFormat.parse(s)
val df = new SimpleDateFormat("dd MMM, yyyy")
println(df.format(date2))
Result:
27 Jan, 2014 res0: Unit = ()
change the
val simpleDateFormat: SimpleDateFormat = new SimpleDateFormat("dd/mm/yyyy")
to
val simpleDateFormat: SimpleDateFormat = new SimpleDateFormat("dd/MM/yyyy")
check this out for all the codes:
http://developer.android.com/reference/java/text/SimpleDateFormat.html
I have a date in this format:
"Fri Oct 31 15:07:24 2014"
and I tried to parse it as I parsed a lot of other dates until now.
I figured out his format is this one (consulting the Java docs (http://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html)):
"EEE MMM dd HH:mm:ss yyyy"
I tried from the scala REPL running this commands:
scala> import java.text.SimpleDateFormat
import java.text.SimpleDateFormat
scala> import java.util.Date
import java.util.Date
scala> val sdf = new SimpleDateFormat("EEE MMM dd HH:mm:ss yyyy")
sdf: java.text.SimpleDateFormat = java.text.SimpleDateFormat#2219f5ee
scala> sdf.parse("Fri Oct 31 15:07:24 2014")
java.text.ParseException: Unparseable date: "Fri Oct 31 15:07:24 2014"
at java.text.DateFormat.parse(DateFormat.java:366)
... 33 elided
but as you can see I get a ParseException.
I tried removing the first part of the Date (and the pattern) like this:
"dd HH:mm:ss yyyy" -> "31 15:07:24 2014"
and all went fine, but when I try to add EEE or MMM I get the ParseException.
I also tried the pattern shown in the java docs that uses EEE and it fails too on my machine.
I've got Java 8 and scala 2.11.1
Thank you in advance.
The problem was the Locale, by Default SimpleDateFormat takes the default Locale of the machine that's running, to set a different Locale (the "en" locale in this example) for the SimpleDateFormat you need to instantiate it this way:
new SimpleDateFormat(format,java.util.Locale.forLanguageTag("en"))