Get specific UTC date and Time in Lua - date

As an illustration, I want to grab the (UTC) +0000 UTC date and time in LUA.
I know there are some answers about this but none of them isn't my answer. The general approach to answering this issue is Find Local Time, then plus or minus the UTC number but I don't know my OS Time Zone because my program cab be run in different locations and I cannot get the timezone from my development environment.
Shortly, How can I get the UTC 0 date and time using LUA functions?

If you need to generate epg, then:
local timestamp = os.time()
local dt1 = os.date( "!*t", timestamp ) -- UTC
local dt2 = os.date( "*t" , timestamp ) -- local
local shift_h = dt2.hour - dt1.hour + (dt1.isdst and 1 or 0) -- +1 hour if daylight saving
local shift_m = 100 * (dt2.min - dt1.min) / 60
print( os.date("%Y%m%d%H%M%S ", timestamp) .. string.format('%+05d' , shift_h*100 + shift_m ))

I calculated the difference in seconds between UTC time and local time. Here's a barebones Time class that applies the shift to store UTC time in from_lua_time. The shift is unapplied by using os.date('!*t', time) which works on UTC time.
-- The number of seconds to add to local time to turn it into UTC time.
-- We need to calculate the shift manually because lua doesn't keep track of
-- timezone information for timestamps. To guarantee reproducible results, the
-- Time class stores epoch nanoseconds in UTC.
-- https://stackoverflow.com/a/43601438/30900
local utc_seconds_shift = (function()
local ts = os.time()
local utc_date = os.date('!*t', ts)
local utc_time = os.time(utc_date)
local local_date = os.date('*t', ts)
local local_time = os.time(local_date)
return local_time - utc_time
end)()
-- Metatable for the Time class.
local Time = {}
-- Private function to create a new time instance with a properly set metatable.
function Time:new()
local o = {}
setmetatable(o, self)
self.__index = self
self.nanos = 0
return o
end
-- Parses the Lua timestamp (assuming local time) and optional nanosec time
-- into a Time class.
function from_lua_time(lua_ts)
-- Clone because os.time mutates the input table.
local clone = {
year = lua_ts.year,
month = lua_ts.month,
day = lua_ts.day,
hour = lua_ts.hour,
min = lua_ts.min,
sec = lua_ts.sec,
isdst = lua_ts.isdst,
wday = lua_ts.wday,
yday = lua_ts.yday
}
local epoch_secs = os.time(clone) + utc_seconds_shift
local nanos = lua_ts.nanosec or 0
local t = Time:new()
local secs = epoch_secs * 1000000000
t.nanos = secs + nanos
return t
end
function Time:to_lua_time()
local t = os.date('!*t', math.floor(self.nanos / 1000000000))
t.yday, t.wday, t.isdst = nil, nil, nil
t.nanosec = self.nanos % 1000000000
return t
end

Related

Scraping Calendar data problem with next day date

Good morning, I have a problem that I do not know how to solve, through web-scraping I take data from the website where I work and create a ics calendar file with my shifts, it works almost always well, only when I have a shift that ends at midnight or later I get an error, this is because in the page I find only the start date and not the end date, so if I finish at 00.00 the correct date should be the next day, but it is added the same as the beginning so I get an error, do you have any idea how to correct this error?
"python nameerror free variable referenced before assignment in enclosing scope"
# CLEANING DATETIME DATUM
datclnd = soup_datum.replace('.', ' ')
dated = tempo.datetime.strptime(datclnd, "%d %m %Y")
yearclnd = dated.year
monthclnd = dated.month
dayclnd = dated.day
# CLEANING DATETIME DATUM
# CLEANING DATETIME HOUR AND MINUTES
time_split = soup_dienstbegin.split(":")
dienstBH = int(time_split[0])
dienstBM = int(time_split[1])
#dienstende
time_split2 = soup_dienstende.split(":")
dienstEH = int(time_split2[0])
try:
dienstEM = int(time_split2[1])
except:
NOTHING
def ics_working():
event = Event()
event['uid'] = f'19970610T172345Z-AF23B2#{dayclnd}{monthclnd}{yearclnd}'
event.add('summary', f'{string1}{tagesinfo2} {soup_kommentar2} {kursinfo}')
event.add('description', f'{schiffinfo}Schichtdauer: {soup_schichtdauer}, Bezahlte Zeit: {soup_bezahltezeit}\n, Infos: {schiffinfo}')
event.add('dtstart', datetime(yearclnd,monthclnd,dayclnd,dienstBH,dienstBM,0))
event.add('dtend', datetime(yearclnd,monthclnd,dayclnd,dienstEH,dienstEM,0))
event.add('dtstamp', datetime(yearclnd,monthclnd,dayclnd,dienstBH,dienstBM,0))
organizer = vCalAddress(f'MAILTO:{receiver_email}')
organizer.params['cn'] = vText(f'{username} Monatsplan')
organizer.params['role'] = vText(f'{username} Monatsplan')
event['organizer'] = organizer
event['location'] = vText('Werftestrasse 5, 6002 Luzern')
# Adding events to calendar
cal.add_component(event)
# Printing working day info
print(f'Datum: {soup_datum} Dienst: {string1}{tagesinfo2} --> Mannschaft: {crew_list2} --> OK')
ics_working()

Qt timestamps from Epoch(1970) to Postgresql timestamps that starts in 2000. How to add an offset and convert from milliseconds to microseconds?

I need to compute the offset from 1970 to 2000 to insert data into postgresql database because postgresql start at 2000 in microseconds resolution and currentMSecsSinceEpoch starts in 1970 in milliseconds. ¿How can an offset be added in order to display the same date in postgresql database?
A solution for computing the offset and converting to microseconds is shown:
QString timestamp = "2000-01-01 00:00:00"; //postgres starts here and uses microseconds resolution
QString timestampEpoch = "1970-01-01 00:00:00"; //epoch starts here, Qt has seconds/milliseconds resolution.
//need to compute the interval between these dates in microseconds
QString format = "yyyy-MM-dd HH:mm:ss";//TZD";
QDateTime origin = QDateTime::fromString(timestampEpoch, format);
QDateTime end = QDateTime::fromString(timestamp, format);
qint64 offset = origin.msecsTo(end);
qint64 timeWithOffset = 1000 * (QDateTime::currentMSecsSinceEpoch() - offset); //mseconds to microseconds for postgresql timestamp(8 bytes)
Hope it helps!

Getting time range between midnight and current time JDK 8

I have this method to calculate midnigt and current time as long values:
/**
* Returns the time range between the midnight and current time in milliseconds.
*
* #param zoneId time zone ID.
* #return a {#code long} array, where at index: 0 - midnight time; 1 - current time.
*/
public static long[] todayDateRange(ZoneId zoneId) {
long[] toReturn = new long[2];
LocalTime midnight = LocalTime.MIDNIGHT;
LocalDate today = LocalDate.now(zoneId);
LocalDateTime todayMidnight = LocalDateTime.of(today, midnight);
ZonedDateTime todayMidnightZdt = todayMidnight.atZone(zoneId);
toReturn[0] = todayMidnightZdt.toInstant().toEpochMilli();
ZonedDateTime nowZdt = LocalDateTime.now().atZone(zoneId);
toReturn[1] = nowZdt.toInstant().toEpochMilli();
return toReturn;
}
Perhaps there is the simpler way to do that?
You could also do:
ZonedDateTime nowZdt = ZonedDateTime.now(zoneId);
ZonedDateTime todayAtMidnightZdt = nowZdt.with(LocalTime.MIDNIGHT);
I can't think of a simpler way to do it.
LocalDateTime vs ZonedDateTime
There's a (tricky) difference between LocalDateTime.now().atZone(zoneId) and ZonedDateTime.now(zoneId).
For the code below, I'm using a JVM in which the default timezone is America/Sao_Paulo and will try to get the current date and time in another timezone (Europe/London). At the moment I run this code, it's August 20th 2017, but in São Paulo the time is 17:56 and in London is 21:56.
When I do:
LocalDateTime nowLdt = LocalDateTime.now();
It creates a LocalDateTime with the current date and time in the JVM's default timezone. In this case, it'll get the current date and time in São Paulo's timezone (which is August 20th 2017, at 17:56):
2017-08-20T17:56:05.159
When I call the atZone method, it creates a ZonedDateTime that corresponds to this date and time in the specified zone:
ZoneId zoneId = ZoneId.of("Europe/London");
ZonedDateTime nowAtZone = nowLdt.atZone(zoneId);
The nowAtZone variable will be:
2017-08-20T17:56:05.159+01:00[Europe/London]
The same date (August 20th 2017) and time (17:56) in London timezone. Note that it's not the current date/time in London. If I get the equivalent epochMilli:
System.out.println(nowAtZone.toInstant().toEpochMilli());
It will be:
1503248165159
Now, if I don't use the LocalDateTime and direclty use the ZonedDateTime instead:
ZonedDateTime nowZdt = ZonedDateTime.now(zoneId);
It will get the current date and time in London, which will be:
2017-08-20T21:56:05.170+01:00[Europe/London]
Note that the time changed (it's 21:56). That's because right now, at this moment, that's the current time in London. If I get the epochMilli value:
System.out.println(nowZdt.toInstant().toEpochMilli());
The value will be:
1503262565170
Note that it's different from the first case using LocalDateTime (even if you ignore the difference in the milliseconds value, because the hour is different). If you want the current date and time at the specified timezone, you must use ZonedDateTime.now(zoneId).
Using LocalDateTime.now().atZone() not only gives a different result, but it will also change if you run in different JVM's, or if the JVM default timezone changes (someone might misconfigure it, or another application running in the same VM calls TimeZone.setDefault()).
Daylight Saving Time
Just remind about corner cases due to DST (Daylight Saving Time) issues. I'm gonna use the timezone I live in as example (America/Sao_Paulo).
In São Paulo, DST started at October 16th 2016: at midnight, clocks shifted 1 hour forward from midnight to 1 AM (and the offset changes from -03:00 to -02:00). So all local times between 00:00 and 00:59 didn't exist in this timezone (you can also think that clocks changed from 23:59:59.999999999 directly to 01:00). If I create a local date in this interval, it's adjusted to the next valid moment:
ZoneId zone = ZoneId.of("America/Sao_Paulo");
// October 16th 2016 at midnight, DST started in Sao Paulo
LocalDateTime d = LocalDateTime.of(2016, 10, 16, 0, 0, 0, 0);
ZonedDateTime z = d.atZone(zone);
System.out.println(z);// adjusted to 2017-10-15T01:00-02:00[America/Sao_Paulo]
When DST ends: in February 19th 2017 at midnight, clocks shifted back 1 hour, from midnight to 23 PM of 18th (and the offset changes from -02:00 to -03:00). So all local times from 23:00 to 23:59 existed twice (in both offsets: -03:00 and -02:00), and you must decide which one you want.
By default, it uses the offset before DST ends, but you can use the withLaterOffsetAtOverlap() method to get the offset after DST ends:
// February 19th 2017 at midnight, DST ends in Sao Paulo
// local times from 23:00 to 23:59 at 18th exist twice
LocalDateTime d = LocalDateTime.of(2017, 2, 18, 23, 0, 0, 0);
// by default, it gets the offset before DST ends
ZonedDateTime beforeDST = d.atZone(zone);
System.out.println(beforeDST); // before DST end: 2018-02-17T23:00-02:00[America/Sao_Paulo]
// get the offset after DST ends
ZonedDateTime afterDST = beforeDST.withLaterOffsetAtOverlap();
System.out.println(afterDST); // after DST end: 2018-02-17T23:00-03:00[America/Sao_Paulo]
Note that the dates before and after DST ends have different offsets (-02:00 and -03:00). This affects the value of epochMilli.
The above can also happen if you adjust the time using with method.
After modification the code now is much simpler:
/**
* Returns the time range between the midnight and current time in milliseconds.
*
* #param zoneId time zone ID.
* #return a {#code long} array, where at index: 0 - midnight time; 1 - current time.
*/
public static long[] todayDateRange(ZoneId zoneId) {
long[] toReturn = new long[2];
//ZonedDateTime nowZdt = LocalDateTime.now().atZone(zoneId);
ZonedDateTime nowZdt = ZonedDateTime.now(zoneId);//As suggested by Hugo (tested).
ZonedDateTime midZdt = nowZdt.with(LocalTime.MIDNIGHT);
toReturn[0] = midZdt.toInstant().toEpochMilli();
toReturn[1] = nowZdt.toInstant().toEpochMilli();
return toReturn;
}

Back and forth UTC dates in lua

I'm having problems converting a lua date to a timestamp and then obtaining back the original date from it. It does work for non UTC dates, but not with UTC.
Currently my example code is:
local dt1 = os.date( "*t" );
print( dt1.hour );
local dt2 = os.date( "*t", os.time( dt1 ) );
print( dt2.hour );
print( "-=-=-" );
local dt1 = os.date( "!*t" );
print( dt1.hour );
local dt2 = os.date( "!*t", os.time( dt1 ) );
print( dt2.hour );
local dt2 = os.date( "*t", os.time( dt1 ) );
print( dt2.hour );
Which yields the output:
12
12
-=-=-
10
9
11
So, in the second part, after obtaining the timestamp using os.time( os.date( "!*t" ) ); I don't know how to obtain the original date back. What is it that I'm doing wrong?
Working with "date tables" in Lua
Let dt be a "date table".
For example, a value returned by os.date("*t") is a "date table".
How to normalize "date table"
For example, after adding 1.5 hours to the current time
local dt = os.date("*t"); dt.min = dt.min + 90
you need to normalize the table fields.
function normalize_date_table(dt)
return os.date("*t", os.time(dt))
end
This function returns new date table which is equivalent to its argument dt regardless of the meaning of content of dt: whether it contains local or GMT time.
How to convert Unix time to "local date table"
dt = os.date("*t", ux_time)
How to convert Unix time to "GMT date table"
dt = os.date("!*t", ux_time)
How to convert "local date table" to Unix time
ux_time = os.time(dt)
How to convert "GMT date table" to Unix time
-- for this conversion we need precalculated value "zone_diff"
local tmp_time = os.time()
local d1 = os.date("*t", tmp_time)
local d2 = os.date("!*t", tmp_time)
d1.isdst = false
local zone_diff = os.difftime(os.time(d1), os.time(d2))
-- zone_diff value may be calculated only once (at the beginning of your program)
-- now we can perform the conversion (dt -> ux_time):
dt.sec = dt.sec + zone_diff
ux_time = os.time(dt)
I found the only solution to works this date , time and time zones is to use custom c-build method :
For example, to retrieve string in wanted format, I use this method:
static int idateformat(lua_State *L) {
time_t t0 = (time_t) lua_tonumber(L,1);
const char* ptrn = lua_tostring(L,2);
const char *tz_value = lua_tostring(L,3);
int isinenv = 0;
if(getenv("TZ") != NULL){
isinenv = 1;
}
char old_tz[64];
if (isinenv == 1) {
strcpy(old_tz, getenv("TZ"));
}
setenv("TZ", tz_value, 1);
tzset();
char new_tz[64];
strcpy(new_tz, getenv("TZ"));
struct tm *lt = localtime(&t0);
//"%Y-%m-%d %H:%M:%S"
char buffer[256];
strftime(buffer, sizeof(buffer), ptrn, lt);
if (isinenv == 1) {
setenv("TZ", old_tz, 1);
tzset();
}
//printf("%ld = %s (TZ=%s)\n", (long)t0, buffer, new_tz);
lua_pushstring(L, buffer);
return 1;
}
Only this way I would suggest.
You're doing nothing wrong.
Lua uses the ISO C and POSIX date and time functions. In particular, os.time uses mktime, which only interprets the input structure according to the current timezone setting in the environment variable TZ. Unfortunately, these standards do not provide a function that interprets the input structure according to GMT/UTC.

Google Bookmark Export date format?

I been working on parsing out bookmarks from an export file generated by google bookmarks. This file contains the following date attributes:
ADD_DATE="1231721701079000"
ADD_DATE="1227217588219000"
These are not standard unix style timestamps. Can someone point me in the right direction here? I'll be parsing them using c# if you are feeling like really helping me out.
Chrome uses a modified form of the Windows Time format (“Windows epoch”) for its timestamps, both in the Bookmarks file and the history files. The Windows Time format is the number of 100ns-es since January 1, 1601. The Chrome format is the number of microseconds since the same date, and thus 1/10 as large.
To convert a Chrome timestamp to and from the Unix epoch, you must convert to seconds and compensate for the difference between the two base date-times (11644473600).
Here’s the conversion formulas for Unix, JavaScript (Unix in milliseconds), Windows, and Chrome timestamps (you can rearrange the +/× and -/÷, but you’ll lose a little precision):
u : Unix timestamp eg: 1378615325
j : JavaScript timestamp eg: 1378615325177
c : Chrome timestamp eg: 13902597987770000
w : Windows timestamp eg: 139025979877700000
u = (j / 1000)
u = (c - 116444736000000) / 10000000
u = (w - 1164447360000000) / 100000000
j = (u * 1000)
j = (c - 116444736000000) / 10000
j = (w - 1164447360000000) / 100000
c = (u * 10000000) + 116444736000000
c = (j * 10000) + 116444736000000
c = (w / 10)
w = (u * 100000000) + 1164447360000000
w = (j * 100000) + 1164447360000000
w = (c * 10)
Note that these are pretty big numbers, so you’ll need to use 64-bit numbers or else handle them as strings like with PHP’s BC-math module.
In Javascript the code will look like this
function chromeDtToDate(st_dt) {
var microseconds = parseInt(st_dt, 10);
var millis = microseconds / 1000;
var past = new Date(1601, 0, 1).getTime();
return new Date(past + millis);
}
1231721701079000 looks suspiciously like time since Jan 1st, 1970 in microseconds.
perl -wle 'print scalar gmtime(1231721701079000/1_000_000)'
Mon Jan 12 00:55:01 2009
I'd make some bookmarks at known times and try it out to confirm.
Eureka! I remembered having read the ADD_DATE’s meaning at some website, but until today, I could not find it again.
http://MSDN.Microsoft.com/en-us/library/aa753582(v=vs.85).aspx
offers this explanation as a “Note” just before the heading “Exports and Imports”:
“Throughout this file[-]format definition, {date} is a decimal integer that represents the number of seconds elapsed since midnight January 1, 1970.”
Before that, examples of {date} were shown:
<DT><H3 FOLDED ADD_DATE="{date}">{title}</H3>
…
and
<DT>{title}
…
Someday, I will write a VBA macro to convert these to recognizable dates, but not today!
If someone else writes a conversion script first, please share it. Thanks.
As of the newest Chrome Version 73.0.3683.86 (Official Build) (64-bit):
When I export bookmark, I got an html file like "bookmarks_3_22_19.html".
And each item has an 'add_date' field which contains date string. like this:
Stack Overflow
This timestamp is actually seconds (not microseconds) since Jan 1st, 1970. So we can parse it with Javascript like following code:
function ChromeTimeToDate(timestamp) {
var seconds = parseInt(timestamp, 10);
var dt = new Date();
dt.setTime(seconds * 1000);
return dt;
}
For the upper example link, we can call ChromeTimeToDate('1553220774') to get Date.
ChromeTimeToDate('1553220774')
12:09:03.263 Fri Mar 22 2019 10:12:54 GMT+0800 (Australian Western Standard Time)
Initially looking at it, it almost looks like if you chopped off the last 6 digits you'd get a reasonable Unix Date using the online converter
1231721701 = Mon, 12 Jan 2009 00:55:01 GMT
1227217588 = Thu, 20 Nov 2008 21:46:28 GMT
The extra 6 digits could be formatting related or some kind of extended attributes.
There is some sample code for the conversion of Unix Timestamps if that is in fact what it is.
look here for code samples: http://www.epochconverter.com/#code
// my groovy (java) code finally came out as:
def convertDate(def epoch)
{
long dv = epoch / 1000; // divide by 1,000 to avoid milliseconds
String dt = new java.text.SimpleDateFormat("dd/MMM/yyyy HH:mm:ss").format(new java.util.Date (dv));
// to get epoch date:
//long epoch = new java.text.SimpleDateFormat("MM/dd/yyyy HH:mm:ss").parse("01/01/1970 01:00:00").getTime() * 1000;
return dt;
} // end of def
So firefox bookmark date exported as json gave me:
json.lastModified :1366313580447014
convert from epoch date:18/Apr/2013 21:33:00
from :
println "convert from epoch date:"+convertDate(json.lastModified)
function ConvertToDateTime(srcChromeBookmarkDate) {
//Hp --> The base date which google chrome considers while adding bookmarks
var baseDate = new Date(1601, 0, 1);
//Hp --> Total number of seconds in a day.
var totalSecondsPerDay = 86400;
//Hp --> Read total number of days and seconds from source chrome bookmark date.
var quotient = Math.floor(srcChromeBookmarkDate / 1000000);
var totalNoOfDays = Math.floor(quotient / totalSecondsPerDay);
var totalNoOfSeconds = quotient % totalSecondsPerDay;
//Hp --> Add total number of days to base google chrome date.
var targetDate = new Date(baseDate.setDate(baseDate.getDate() + totalNoOfDays));
//Hp --> Add total number of seconds to target date.
return new Date(targetDate.setSeconds(targetDate.getSeconds() + totalNoOfSeconds));
}
var myDate = ConvertToDateTime(13236951113528894);
var alert(myDate);
//Thu Jun 18 2020 10:51:53 GMT+0100 (Irish Standard Time)
#Python program
import time
d = 1630352263 #for example put here, if (ADD_DATE="1630352263")
print(time.ctime(d)) #Mon Aug 30 22:37:43 2021 - you will see