'04/30/2019' asDate works, but '30/04/2019' asDate fails because the date format of this date is in the 'dd/MM/yyyy' format.
How do I specify different time format in Pharo 8?
There are several ways to get a Date from a String. In your case, one that would work is the following
Date readFrom: '30/4/2019' readStream pattern: 'dd/m/yyyy'
The 'm' in the pattern will match two digit month indexes too. If you use 'mm' instead your month indexes must have two digits, e.g., '04'.
There is nothing similar for DateAnTime. However you can do the following:
| stream |
stream := '30/4/2019 18:11:03' readStream.
date := Date readFrom: (stream upTo: $ ) readStream pattern: 'dd/m/yyyy'.
time := Time readFrom: stream.
^DateAndTime date: date time: time.
This uses the first part of the stream (up to the space) for the date and then continues with the time. Note that the stream is left at the character next to the space, which should be where the time begins.
Related
I'm working on tables obtained from a PervasiveSQL database and I have some trouble managing dates.
In some of the fields dates are recorded in the format we use in Italy, dd/mm/yyyy, but in others are recorded in a format I can't understand, something like this:
Start_Date 132384788
Last_Tx_Date 132385052
Last_Tx_Time 252711936
What kind of format is it?
How can I convert it in a human readable one?
I think that Start_Date could be August 8 2020 but I'm not sure.
Thanks for any help!
I tried to copy and paste tables in an Excel file but automatic dates conversion did not work.
The Start_Date and Last_Tx_Date fields look to be Btrieve Date fields. If you set the data type for that field in the DDFs to Date, it should show a human readable field. However the Last_Tx_Time field is a Btrieve Time (not timestamp) type.
From the Actian Zen v15.10 documentation (https://docs.actian.com/zen/v15/#page/sqlref/sqldtype.htm#ww136646):
Date:
The DATE key type is stored internally as a 4-byte value. The day and the month are each stored in 1-byte binary format. The year is a 2-byte binary number that represents the entire year value. The MicroKernel places the day into the first byte, the month into the second byte, and the year into a two-byte word following the month.
An example of C structure used for date fields would be:
TYPE dateField {
char day;
char month;
integer year;
}
The year portion of a date field is expected to be set to the integer representation of the entire year. For example, 2,001 for the year 2001.
Time:
The TIME key type is stored internally as a 4-byte value. Hundredths of a second, second, minute, and hour values are each stored in 1-byte binary format. The MicroKernel places the hundredths of a second value in the first byte, followed respectively by the second, minute, and hour values. The data format is hh:mm:ss.nn. Supported values range from 00:00:00.00 to 23:59:59.99.
I have a spark data frame having two columns (SEQ - Integer, MAIN_DATE - Date) as:
Now I want to add a column based on the condition that if the format of MAIN_DATE is "MMM-YYYY" then it should be converted to Last day of the month and new data frame should look like this:
Any suggestion will be much appreciated.
You can use Spark's when/otherwise methods in order to operate differently for each different date format of the MAIN_DATE column.
More specifically, you can simply match the MMM-yyyy date format values of the column based on the field's String length (since we know that those values we always have 8 characters) as a condition in when and then:
use to_date to convert the String value to a valid date based on a format we give as an argument, and
use last_date to get the last day of the month each curry date in MAIN_DATE is referring to.
As for the "regular" rows with the dd-MMM-yyyy date format, just a to_date conversion would be sufficient within the otherwise method.
After that, all there's left to do is to convert the dates back to the desired dd-MMM-yyyy format (because to_date converts a given date to the yyyy-MM-dd format).
This is the solution in Scala (split in into two withColumns to make it more readable, instead of an one-liner):
df.withColumn("END_DATE",
when(length(col("MAIN_DATE")).equalTo(8), last_day(to_date(col("MAIN_DATE"), "MMM-yyyy")))
.otherwise(to_date(col("MAIN_DATE"), "dd-MMM-yyyy")))
.withColumn("END_DATE", date_format(col("END_DATE"), "dd-MMM-yyyy"))
This is what the resulting df DataFrame will look like:
+---+-----------+-----------+
|SEQ| MAIN_DATE| END_DATE|
+---+-----------+-----------+
| 1|16-JAN-2020|16-Jan-2020|
| 2| FEB-2017|28-Feb-2017|
+---+-----------+-----------+
I have been trying to upload a table with dates on the from a csv file but I keep getting an error about the date type like this:
Errors:
Too many errors encountered. (error code: invalid)
query: Invalid date: '2010-06-31' (error code: invalidQuery)
So it is complaining about 2010-06-31. I checked the reference and it says:
Date type
Name Description DATE Represents a logical calendar date. Values range
between the years 1 and 9999, inclusive. The DATE type represents a
logical calendar date, independent of time zone. A DATE value does not
represent a specific 24-hour time period. Rather, a given DATE value
represents a different 24-hour period when interpreted in different
time zones, and may represent a shorter or longer day during Daylight
Savings Time transitions. To represent an absolute point in time, use
a timestamp.
Canonical format
'YYYY-[M]M-[D]D' YYYY: Four-digit year [M]M: One or two digit month
[D]D: One or two digit day
https://cloud.google.com/bigquery/sql-reference/data-types#date-type
It says YYYY-[M]M-[D]D so I thought 2010-06-31 is correct but still getting an error.
My rows look like this in the csv file:
Regular Season,2010-06-31,Chicago,Road,22,37,21,28,,,,,108,240,39,79
My schema looks like this:
_Dataset: STRING
_DATE: DATE
_TEAMS: STRING
_VENUE: STRING
_1Q: INTEGER
_2Q: INTEGER
_3Q: INTEGER
_4Q: INTEGER
_OT1: INTEGER
_OT2: INTEGER
_OT3: INTEGER
_OT4: INTEGER
_F: INTEGER
_MIN: INTEGER
_FG: INTEGER
_FGA: INTEGER
Thanks in advance for your help
Even though June 31 exists as per The Thirty-first of June by J.B. Priestley -
your issue can be simply just because in reality - June month has only 30 days, so load engine gets stuck with June 31st
On the other hand - query engine successfully "translates" 2010-06-31 into 2010-07-01 - try below example
SELECT DATE('2010-06-31')
So I have one big file (13 million rows) and date formatted as:
2009-04-08T01:57:47Z. Now I would like to split it into 2 columns now,
one with just date as dd-MM-yyyy and other with time only hh:MM.
How do I do it?
You can simply use tMap and parseDate/formatDate to do what you want. It is neither necessary nor recommended to implement your own date parsing logic with regexes.
First of all, parse the timestamp using the format yyyy-MM-dd'T'HH:mm:ss'Z'. Then you can use the parsed Date to output the formatted date and time information you want:
dd-MM-yyyy for the date
HH:mm for the time (Note: you mixed up the case in your question, MM stands for the month)
If you put that logic into a tMap:
you will get the following:
Input:
timestamp 2009-04-08T01:57:47Z
Output:
date 08-04-2009
time 01:57
NOTE
Note that when you parse the timestamp with the mentioned format string (yyyy-MM-dd'T'HH:mm:ss'Z'), the time zone information is not parsed (having 'Z' as a literal). Since many applications do not properly set the time zone information anyway but always use 'Z' instead, so this can be safely ignored in most cases.
If you need proper time zone handling and by any chance are able to use Java 7, you may use yyyy-MM-dd'T'HH:mm:ssXXX instead to parse your timestamp.
I'm guessing Talend is falling over on the T and Z part of your date time stamp but this is easily resolved.
As your date time stamp is in a regular pattern we can easily extract the date and time from it with a tExtractRegexFields component.
You'll want to use "^([0-9]{4}-[0-9]{2}-[0-9]{2})T([0-9]{2}:[0-9]{2}):[0-9]{2}Z" as your regex which will capture the date in yyyy-MM-dd format and the time as mm:HH (you'll want to replace the date time field with a date field and a time field in the schema).
Then to format your date to your required format you'll want to use a tMap and use TalendDate.formatDate("dd-MM-yyyy",TalendDate.parseDate("yyyy-MM-dd",row7.date)) to return a string in the dd-MM-yyyy format.
I am developing a plugin written in Lua, and I need a way to calculate Unix time or at least a way to compare 2 date strings.
The function I can use only returns date string in the following format
"1/17/2014 6:50 PM"
Is there a way to convert this string to a Unix time?
Unfortunately I don't have access to the OS library so things like os.time() do not work.
Is there any library or something similar that I can work with?
I also thought about splitting the string into parts, but I need a way to add/subtract time
Just compare normalized timestamps:
function normalize(a)
local m,d,y,h,mi,n=a:match("(%d+)/(%d+)/(%d+)%s+(%d+):(%d+)%s+(%w+)")
if n=="PM" then h=h+12 end
return string.format("%04d%02d%02d%02d%02d",y,m,d,h,mi)
end
Date arithmetic is another story. For a complete, pure Lua date library, see luatz or https://github.com/Tieske/date.
If you need to only compare two time, you don't need to get each time's Unix timestamp. One possible solution is to get the time fields from the string like this:
local time = "1/17/2014 6:50 PM"
local month, day, year, hour, minute, am_pm = time:match("(%d+)/(%d+)/(%d+)%s+(%d+):(%d+)%s+(%w+)")
print(month, day, year, hour, minute, am_pm)
Output: 1 17 2014 6 50 PM
Then compare two time from comparing their year, if they are equal, then month, and so on. Remember to use tonumber to compare them by number, not the string itself.