How do I convert Epoch time to Date in Open Refine? - date

I don't care which language I use (as long as it's one of the three available in Open Refine), but I need to convert a timestamp returned from an API from epoch time to a regular date (see Expression box in the screenshot below). Not too picky about the output date format, just that it retains the date down to the second. Thanks!
Can use: GREL, Jython, or Clojure.

If you have to stick to GREL you can use the following one-liner:
inc(toDate("01/01/1970 00:00:00","dd/MM/YYYY H:m:s"),value.toNumber(),"seconds").toString('yyyy-MM-dd HH:mm:ss')
Breaking it down:
inc(date d, number value, string unit) as defined in the GREL documentation : Returns a date changed by the given amount in the given unit of time. Unit defaults to 'hour'
toDate(o, string format) : Returns o converted to a date object. (more complex uses of toDate() are shown in the GREL documentation)
We use the string "01/01/1970 00:00:00" as input for toDate() to get the start of the UNIX Epoch (January 1st 1970 midnight).
We pass the newly created date object into inc() and as a second parameter the result of value.toNumber() (assuming value is a string representation of the number of seconds since the start of the Unix Epoch), as a 3rd parameter, the string "seconds" which tells inc() the unit of the 2nd parameter.
We finally convert the resulting date object into a string using the format: yyyy-MM-dd HH:mm:ss
Test Data
Following is a result of using the function described above to turn a series of timestamps grabbed from the Timestamp Generator into string dates.
| Name | Value | Date String |
|-----------|------------|---------------------|
| Timestamp | 1491998962 | 2017-04-09 12:09:22 |
| +1 Hour | 1492002562 | 2017-04-09 13:09:22 |
| +1 Day | 1492085362 | 2017-04-10 12:09:22 |
| +1 Week | 1492603762 | 2017-04-16 12:09:22 |
| +1 Month | 1494590962 | 2017-05-09 12:09:22 |
| +1 Year | 1523534962 | 2018-04-09 12:09:22 |

Unfortunately, I do not think you can do it with a GREL statement like this or somesuch, but I might be pleasantly surprised by someone else that can make it work somehow:
value.toDate().toString("dd/MM/yyy")
So in the meantime, use this Jython / Python Code:
import time;
# This is a comment.
# We change 'value' to an integer, since time needs to work with numbers.
# If we needed to, we could also * 1000 if we had a Unix Epoch Time in seconds, instead of milliseconds.
# We also have no idea what the local time zone is for this, which could affect the date. But we digress...
epochlong = int(float(value));
datetimestamp = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(epochlong));
return datetimestamp

Related

How to subtract date only format "mm/dd" in excel?

My date data was pulled from our system with the format "mm/dd" not the year.
So I meet the problem when I subtract value between the old year and the current year.
Example:
Date action Date Check Result Current Date
12/21 01/03 -352 03/18/2022
The correct result is:
Date action Date Check Result Current Date
12/21 01/03 13 03/18/2022
How to subtract correctly? Thanks.
I assume that Excel has treated 12/21 and 01/03 as dates, but in doing this has assumed the current year in all cases. This dedcuts 1 year from cell A2
=DATE(YEAR(A2)-1,MONTH(A2),DAY(A2))
e.g.
Test if the earlier date has a month less than the latter date, if it is not then deduct 1 year from the earlier date and then calculate the difference
+------------+------------+------+
| A | B | C |
1 | earlier | latter | diff |
+------------+------------+------+
2 | 2022-12-21 | 2022-01-03 | 13 |
+------------+------------+------+
in cell C2
=IF(MONTH(A2) > MONTH(B2),B2-DATE(YEAR(A2)-1,MONTH(A2),DAY(A2)),B2-A2)

How to parse month-year string using Presto

I have a column that contains a Month-Year string that I would like to convert to an actual date representing the first day of the Month and Year combination. For example
+----------+------------+
| Original | Desired |
+----------+------------+
| Aug-19 | 08/01/2019 |
+----------+------------+
| Sep-20 | 09/01/2020 |
+----------+------------+
| May-22 | 05/01/2022 |
+----------+------------+
I have tried breaking apart the Month-Year string using split_part but when I try and pass Month as a parameter into date_parse it throws an error with the input (INVALID_FUNCTION_ARGUMENT). I could break apart the Month-Year into strings and then recombine, hard-coding the 01 however the problem seems that three letter month cannot be parsed into an actual month by Presto. I also want to avoid a 12 line CASE WHEN statement to parse the month if possible.
I'm not sure where the year comes from, but the query will be like this:
select date_format(date_parse('May-22', '%b-%d'), '%m/%d/%Y')
https://trino.io/docs/current/functions/datetime.html?mysql-date-functions

Convert variable of type double to date and time

How do I do the following in Stata:
I have a variable of type double with values similar to the ones below:
20180405013331
20160107085521
How can I convert it to a date/time (YYYYMMDDhhmmss) variable like the following:
2018April5 01:33:31
2016January7 08:55:21
help datetime explains the basics here. The only extra twist is that your date-times arrive as a double, so you need to convert to a string, which can be done on the fly.
clear
input double mydate
20180405013331
20160107085521
end
format mydate %14.0f
gen double wanted = clock(string(mydate, "%14.0f"), "YMDhms")
format wanted %tc
list
+-------------------------------------+
| mydate wanted |
|-------------------------------------|
1. | 20180405013331 05apr2018 01:33:31 |
2. | 20160107085521 07jan2016 08:55:21 |
+-------------------------------------+

How should I store Go's time.Location in Postgres?

In Postgres I store data given to me by a user with:
Column | Type | Collation | Nullable | Default
------------+--------------------------+-----------+----------+---------
id | uuid | | not null |
value | numeric | | |
date | timestamp with time zone | | |
Now I'm presented with the requirement of maintaining the original timezone in which the data was produced. The timestamp with timezone is normalized to the database's timezone and the original timezone is lost, so I must manually restore date back from the normalized timezone before serving it back to the user.
Most solutions suggest adding an extra column to the table and storing the original timezone information together with the timestamp:
Column | Type | Collation | Nullable | Default
------------+--------------------------+-----------+----------+---------
id | uuid | | not null |
value | numeric | | |
date | timestamp with time zone | | |
tz | text | | |
So given that I'm using Go, which information should I extract from time.Time to store in tz for the most precise and seamless restoration?
date.Location().String() doesn't seem right as it might return the value Local which is relative.
And how should I restore the information from tz back into to time.Time?
Is the result of time.LoadLocation(tz) good enough?
Upon save, I would obtain the zone name and offset using Time.Zone():
func (t Time) Zone() (name string, offset int)
Then when querying such a timestamp from the database, you can construct a time.Location using time.FixedZone():
func FixedZone(name string, offset int) *Location
And switch to this location using Time.In().
Word of caution! This will restore you a timestamp with "seemingly" in the same time zone, but if you need to apply operations on it (such as adding days to it), the results might not be the same. The reason for this is because time.FixedZone() returns a time zone with a fixed offset, which does not know anything about daylight savings for example, while the original timestamp you saved might have a time.Location which does know about these things.
Here's an example of such a deviation. There is a daylight saving day in March, so we'll use a timestamp pointing to March 1, and add 1 month to it, which results in a timestamp being after the daylight saving.
cet, err := time.LoadLocation("CET")
if err != nil {
panic(err)
}
t11 := time.Date(2019, time.March, 1, 12, 0, 0, 0, cet)
t12 := t11.AddDate(0, 1, 0)
fmt.Println(t11, t12)
name, offset := t11.Zone()
cet2 := time.FixedZone(name, offset)
t21 := t11.UTC().In(cet2)
t22 := t21.AddDate(0, 1, 0)
fmt.Println(t21, t22)
now := time.Date(2019, time.April, 2, 0, 0, 0, 0, time.UTC)
fmt.Println("Time since t11:", now.Sub(t11))
fmt.Println("Time since t21:", now.Sub(t21))
fmt.Println("Time since t12:", now.Sub(t12))
fmt.Println("Time since t22:", now.Sub(t22))
This will output (try it on the Go Playground):
2019-03-01 12:00:00 +0100 CET 2019-04-01 12:00:00 +0200 CEST
2019-03-01 12:00:00 +0100 CET 2019-04-01 12:00:00 +0100 CET
Time since t11: 757h0m0s
Time since t21: 757h0m0s
Time since t12: 14h0m0s
Time since t22: 13h0m0s
As you can see, the output time after the 1-month addition is the same, but the zone offset is different, so they designate a different time instant in time (which is proven by showing the time difference with an arbitrary time). The original has 2-hour offset, because it knows about the daylight saving that happened in the 1 month we skipped, while the "restored" timestamp's zone doesn't know about that, so the result has the same 1-hour offset. In the timestamp after addition, even the zone name changes in real life: from CET to CEST, and again, the restored timestamp's zone doesn't know about this either.
A more wasteful, and still prone to error, but still valid solution will be to also store the original timestamp in ISO 8601 format like 2019-05-2T17:24:37+01:00 in a separate column datetext:
Column | Type | Collation | Nullable | Default
------------+--------------------------+-----------+----------+---------
id | uuid | | not null |
value | numeric | | |
date | timestamp with time zone | | |
datetext | text | | |
Then query using date for the strength of the native timestamp column, and return to the user datetext which is the exact value that was originally sent.

Calculate time range in org-mode table

Given a table that has a column of time ranges e.g.:
| <2015-10-02>--<2015-10-24> |
| <2015-10-05>--<2015-10-20> |
....
how can I create a column showing the results of org-evalute-time-range?
If I attempt something like:
#+TBLFM: $2='(org-evaluate-time-range $1)
the 2nd column is populated with
Time difference inserted
in every row.
It would also be nice to generate the same result from two different columns with, say, start date and end date instead of creating one column of time ranges out of those two.
If you have your date range split into 2 columns, a simple subtraction works and returns number of days:
| <2015-10-05> | <2015-10-20> | 15 |
| <2013-10-02 08:30> | <2015-10-24> | 751.64583 |
#+TBLFM: $3=$2-$1
Using org-evaluate-time-range is also possible, and you get a nice formatted output:
| <2015-10-02>--<2015-10-24> | 22 days |
| <2015-10-05>--<2015-10-20> | 15 days |
| <2015-10-22 Thu 21:08>--<2015-08-01> | 82 days 21 hours 8 minutes |
#+TBLFM: $2='(org-evaluate-time-range)
Note that the only optional argument that org-evaluate-time-range accepts is a flag to indicate insertion of the result in the current buffer, which you don't want.
Now, how does this function (without arguments) get the correct time range when evaluated is a complete mystery to me; pure magic(!)