is there a method to convert the unit from hours to days in this dataset ?
double time(time) ;
time:units = "hours since 1800-01-01 00:00:0.0" ;
time:long_name = "Time" ;
time:delta_t = "0000-01-00 00:00:00" ;
time:avg_period = "0000-01-00 00:00:00" ;
time:standard_name = "time" ;
time:axis = "T" ;
time:actual_range = 1569072., 1895592. ;
If you can use Python, it's an easy process:
The first step is to convert the numeric dates to a datetime object using netCDF4 num2date.
The second step is to compute the number of days between each datetime object and the time stamp (or original date) in the time variable (i.e. 1800-01-01).
import netCDF4
import datetime
ncfile = netCDF4.Dataset('./precip.mon.mean.nc', 'r')
time = ncfile.variables['time']
# Convert from numeric times to datetime objects
dates = netCDF4.num2date(time[:], time.units)
# Compute number of days since the original date
orig_date = datetime.datetime(1800,1,1)
days_since = [(t - orig_date).days for t in dates]
Related
I'm trying to read a parquet file using boto3. The original file has dates with the following format:
2016-12-07 23:00:00.000
And they are stored as timestamps.
My code in Sage Maker is:
boto_s3 = boto3.client('s3')
r = boto_s3.select_object_content(
Bucket='bucket_name',
Key='path/file.gz.parquet',
ExpressionType='SQL',
Expression=f"select fecha_instalacion,pais from s3object s ",
InputSerialization = {'Parquet': {}},
OutputSerialization = {'CSV': {}},
)
rl0 = list(r['Payload'])[0]
from io import StringIO
string_csv = rl0['Records']['Payload'].decode('ISO-8859-1')
csv = StringIO(string_csv)
pd.read_csv(csv, names=['fecha_instalacion', 'pais'])
But instead of the date I get:
fecha_instalacion pais
45352962065516692798029824 ESPAÃA
I loooked for dates with only one day in between and the nyuumber of digits that are the same are the first 6. As an example:
45337153205849123712294912--> 2016-12-09 23:00:00.000
45337116312360976293191680--> 2016-12-07 23:00:00.000
I would need to get the correct formated date, and avoid the especial characters.
Thanks.
The problem is the format. That Parquet file is using Int96 numbers to represent timestamp.
Here is a function to convert the int96Timestamp to python DateTime
import datetime
def dateFromInt96Timestamp(int96Timestamp):
julianCalendarDays = int96Timestamp >> 8*8
time = int((int96Timestamp & 0xFFFFFFFFFFFFFFFF) / 1_000)
linuxEpoch = 2_440_588
return datetime.datetime(1970, 1, 1) + datetime.timedelta(days=julianCalendarDays - linuxEpoch, microseconds=time)
I have a variable period that contains a month as an abbreviated string (i.e. "JAN", "FEB", "MAR", etc). How do I convert period to a numeral (i.e. 1, 2, 3, etc)?
My solution is:
gen fake_date_s = "2000"+period+"1"
gen fake_date = date(fake_date_s, "YMD")
gen month = month(fake_date)
I don't think it's ugly:
clear
input ///
str3 period
JAN
FEB
DEC
end
list
gen monthnum = month(date("2000" + period + "1", "YMD"))
list
This also works:
gen monthnum = month(date(period, "M"))
as it sets the day and the year in the daily date to 01 and 1960, by default.
I'm sure you can find an alternative that doesn't use date functions, but why not use them?
Another approach is:
local i=1
foreach m in `c(Mons)' {
replace month = "`i'" if month == upper("`m'")
local ++i
}
destring month, replace
This is my code so far:
import datetime
from time import strptime
leapyear = 0
isValid = False
while not isValid:
in_date = input(" Please in put a year in the format dd/mm/yyyy ")
try:
d = strptime(in_date, '%d/%m/%Y')
isValid=True
except:
print ("This is not in the right format")
date = datetime.date.today().strftime("%d/""%m/""%Y")
in_date = in_date.split('/')
date = date.split('/')
in_date = [int(i) for i in in_date]
date = [int(i) for i in date]
date_f = [str(i) for i in date]
date_f = '/'.join(date_f)
in_date_f = [str(i) for i in in_date]
in_date_f = '/'.join(in_date_f)
newdate = []
in_date[0], in_date[2] = in_date[2], in_date[0]
date[0], date[2] = date[2], date[0]
z = "/"
if in_date > date:
newdate.insert((0),(in_date[2] % date[2]))
newdate.insert((1),(in_date[1] % date[1]))
newdate.insert((2),(in_date[0] % date[0]))
print("Current Date:", date_f)
print("you are:", newdate[2],"year(s)",newdate[1],"month(s) and",newdate[0],"days away from:",in_date_f)
else:
print("Please input a date thats higher than todays.")
At the moment i have taken today's date and then taken away that from the user input date which has to be higher than the current date. but this gives the wrong answer because it hasnt taken in the fact of the days in a month the and the months in a year.
how would i go about doing that?
datetime objects are subtractable and comparable, so there is no problem in doing:
userDate = strptime(in_date, '%d/%m/%Y')
if userDate > datetime.datetime.today():
# ...
or
if userDate.date() > datetime.datetime.today().date():
# ....
or to calculate the difference:
diff = userDate - datetime.datetime.today()
I computed the difference of two ISO 8601 dates after coverting them to epoch.
How can I get the difference of them in number of days?
My code is
my $ResolvedDate = "2014-06-04T10:48:07.124Z";
my $currentDate = "2014-06-04T06:03:36-04:00"
my $resolved_epoch = &convert_time_epoch($ResolvedDate);
my $current_epoch = &convert_time_epoch($currentDate);
if (($resolvedDate - $currentDate) > $noOfDays) {
print "Difference in greater than x\n";
$built = 0;
return ($built);
} else {
print "Difference in smaller than x \n";
$built = 1;
return ($built);
}
sub convert_time_epoch {
my $time_c = str2time(#_);
my #time_l = localtime($time_c);
my $epoch = strftime("%s", #time_l);
return($epoch);
}
Here in addition to $built I also want to return exact number of days, Resolved date is greater than Current date.
"number of days" is awkward, because this is localtime and DST exists (or at least, may exist).
By simply dividing by 86400 you can easily obtain the number of 24-hour periods, which may be sufficient for your purposes.
If you want the true number of times that the mday field has changed, this may be slightly different from the value obtained by this simple division, however.
If the dates are in epoch seconds, take the difference and divide it by the number of seconds in a day (which is 86400). Like so:
my $days_difference = int(($time1 - $time2) / 86400);
If you use DateTime then
my $duration = $dt1->delta_days($dt2); #$dt1 and $dt2 are DateTime objects.
print $duration->days;
use DateTime::Format::ISO8601 qw( );
my $ResolvedDate = "2014-06-04T10:48:07.124Z";
my $currentDate = "2014-06-04T06:03:36-04:00";
my $format = DateTime::Format::ISO8601->new();
my $dt_resolved = $format->parse_datetime($ResolvedDate);
my $dt_current = $format->parse_datetime($currentDate);
my $dur = $dt_resolved->delta_days($dt_current);
my $days = $dur->in_units('days');
I have a text file that contains timestamps out of a camera that captures 50 frames per second .. The data are as follows:
1 20931160389
2 20931180407
3 20931200603
4 20931220273
5 20931240360
.
.
50 20932139319
... and so on.
It gives also the starting time of capturing like
Date: **02.03.2012 17:57:01**
The timestamps are in microseconds not in milliseconds, and MATLAB can support only till milliseconds but its OK for me.
Now I need to know the human format of these timestamps for each row..like
1 20931160389 02.03.2012 17:57:01.045 % just an example
2 20931180407 02.03.2012 17:57:01.066
3 20931200603 02.03.2012 17:57:01.083
4 20931220273 02.03.2012 17:57:01.105
5 20931240360 02.03.2012 17:57:01.124
and so on
I tried this:
%Refernce Data
clc; format longg
refTime = [2012,03,02,17,57,01];
refNum = datenum(refTime);
refStr = datestr(refNum,'yyyy-mm-dd HH:MM:SS.FFF');
% Processing data
dn = 24*60*60*1000*1000; % Microseconds! I have changed this equation to many options but nothing was helpful
for i = 1 : size(Data,1)
gzTm = double(Data{i,2}); %timestamps are uint64
gzTm2 = gzTm / dn;
gzTm2 = refNum + gzTm2;
gzNum = datenum(gzTm2);
gzStr = datestr(gzNum,'yyyy-mm-dd HH:MM:SS.FFF'); % I can't use 'SS.FFFFFF'
fprintf('i = %d\t Timestamp = %f\t TimeStr = %s\n', i, gzTm, gzStr);
end;
But I got always strange outputs like
i = 1 Timestamp = 20931160389.000000 TimeStr = **2012-03-08 13:29:28.849**
i = 2 Timestamp = 20931180407.000000 TimeStr = **2012-03-08 13:29:29.330**
i = 3 Timestamp = 20931200603.000000 TimeStr = **2012-03-08 13:29:29.815**
The output time is about some hours late/earlier than the Referenced Time. The day is different.
The time gap between each entry in the array should be nearly 20 seconds..since I have 50 frames per second(1000 millisecond / 50 = 20) ..and the year,month, day,hour,minute and seconds should also indicate the initial time given as reference time because it is about some seconds earlier.
I expect something like:
% just an example
1 20931160389 02.03.2012 **17:57:01.045**
2 20931180407 02.03.2012 **17:57:01.066**
Could one help me please..! Where is my mistake?
It looks like you can work out the number of microseconds between a record and the first record:
usecs = double(Data{i,2}) - double(Data{1,2});
convert that into seconds:
secsDiff = usecs / 1e6;
then add that to the initial datetime you'd calculated:
matDateTime = refNum + secsDiff / (24*60*60);