Is there a perl function that converts YYYYMMDD to a 16digit epoch time?
I'm currently calling the date command:
date -d 20160219 +%s%6N
for the conversion but my script takes a long time to go through the millions of dates in my data set.
By "16digit epoch time", I mean a 16-digit decimal integer representing microseconds since the epoch. For example, 2016-02-19 00:00:00 UTC is 1455840000 seconds after the epoch, so the result I want is "1455840000000000".
You can also use Time::Moment. In the interest of full disclosure, I am the author of Time::Moment.
use v5.10;
use Time::Moment;
say Time::Moment->from_string('20160219' . 'T00Z')
->strftime('%s%6N');
Output:
1455840000000000
Time::Piece can do all the things you want. After that just right pad 0's to epoch, so that it becomes 16 digits.
#!/usr/bin/perl
use strict;
use warnings;
use Time::Piece;
my $t = Time::Piece -> strptime("20160219",
"%Y%m%d");
my $epoch = $t -> epoch;
# make $epoch 16 digits
$epoch .= "0" x (16 - length($epoch));
print "$epoch\n";
You can use the Time::Local core module:
use Time::Local 'timelocal';
$date = "20160218";
($year, $month, $day) = unpack "A4A2A2", $date;
$time = timelocal(0, 0, 0, $day, $month-1, $year);
$time .= '000000';
Wow, I just actually wrote a script to take the timestamp where it takes a timestamp where it's in seconds since epoch and convert it to human readable format. Not what you asked, but perhaps you can reverse what I wrote.
my $ts = $ARGV[0]; #2012-09-04-20:24:19.870 1030825460
my $newTs = toTimeString($ts);
print "\n".$newTs."\n";
## Convert from time since epoch in seconds to string YYYY-MM-DD-HH:mm:ss.sss
sub toTimeString{
my $ts=shift;
$ts += 432000; # Add 5 days to make computation easy.
my $sec = $ts-int($ts/60)*60; # Seconds
$ts = int($ts/60); # Convert to minutes
my $min = $ts%60; # minutes
$ts = int($ts/60); # Convert to hours.
my $hr = $ts%24; # hours;
$ts = int($ts/24); # Convert to days.
my $yr;
my $mo;
my $day;
my $days = [ [31,28,31,30,31,30,31,31,30,31,30,31],
[31,29,31,30,31,30,31,31,30,31,30,31]
];
our $cumDays = [ [0,31,59,90,120,151,181,212,243,273,304,334],
[0,31,60,91,121,152,182,213,244,274,305,335]
];
if($ts<366){$yr=1980;}
else{
$yr = 1981 + int(($ts-366)/1461)*4;
$ts = $ts - 366 - int(($ts-366)/1461)*1461;
if($ts < 1095){$yr += int($ts/365); $ts = $ts - int($ts/365)*365;}
else{$yr = $yr+3; $ts=$ts-1095;}
}
my $leap=0;
my $i;
if($yr%4 == 0){$leap=1;}
for($i=0;$i<12;$i++){
if($ts > $cumDays->[$leap]->[$i]){
$mo=$i+1;
}
}
$day=$ts-$cumDays->[$leap]->[$mo-1]+1;
my $ret;
$ret = sprintf("%04d",$yr)."-".sprintf("%02d",$mo)."-".sprintf("%02d",$day)."-";
$ret = $ret.sprintf("%02d",$hr).":".sprintf("%02d",$min).":".sprintf("%02d",int($sec));
$ret = $ret."\.".sprintf("%03d",int($sec*1000)%1000);
return ($ret)
}
Related
PERL : I take mday value from locatime as below ,now I want value of day before yesterday . How can I subtract 1 from mday which I take from localtime
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime();
my $part = "P".$mday;
print "Today value is $part \n";
my $part_yes = "P".$mday - $num;
print "$part_yes \n";
Using DateTime:
my $dt =
DateTime
->now( time_zone => 'local' )
->set_time_zone('floating') # Do this when working with dates.
->truncate( to => 'days' ); # Optional.
$dt->subtract( days => 2 );
my $yesterday = $dt->day;
DateTime is pretty heavy, and it seems people asking date-time questions invariably come back and say "core modules only!", so here's a solution using only core modules.
use Time::Local qw( timegm );
# Create an timestamp with the same date in UTC as the one local one.
my $epoch = timegm(0, 0, 0, ( localtime() )[3,4,5]);
# We can now do date arithmetic without having to worry about DST switches.
$epoch -= 2 * 24*60*60;
my $yesterday = ( gmtime($epoch) )[3] + 1;
my $now = time(); # time as seconds since 1970-01-01
my $day_ago = $now - 24*60*60;
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($day_ago);
my $part = "P".$mday;
WARNING (ikegami comment): Not all days have 24 hours. This can produce a result that's 0, 1 or 2 days earlier
https://perldoc.perl.org/functions/time
https://perldoc.perl.org/functions/localtime
My script is :
use warnings;
use strict;
my $start_time = localtime();
$sql = "INSERT into Table1, Select (VAr1, Var2, Var3 ...........)";
my $end_time = localtime();
my $run_time = ($end_time - $start_time);
my #months = ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec");
my ($sec, $min, $hour, $day,$month,$year) = (localtime($run_time))[0,1,2,3,4,5];
print "Unix time ".$run_time." converts to ".$months[$month]." ".$day.", ".($year +=1900);
print " ".$hour.":".$min.":".$sec."\n";
When I run it, I don't get the desire output.
Unix time 648 converts to Jan , 1900 ::
I need to get hour:min:sec
the time the script took to calculate the insert into the table.
Thank you
$run_time isn't a timestamp (a number of seconds since epoch representing a date-time), so it makes no sense to pass it to localtime.
my $s = $run_time % 60; $run_time = ($run_time - $s) / 60;
my $m = $run_time % 60; $run_time = ($run_time - $m) / 60;
my $h = $run_time;
my $formatted_run_time = sprintf "%d:%02d:%02d", $h, $m, $s;
You can get fancier too:
my $formatted_run_time =
$h ? sprintf "%d:%02d:%02d", $h, $m, $s
$m ? sprintf "%d:%02d", $m, $s
$s;
localtime is used to convert unix timestamps to a human-readable time (or a list of parts of it), but you are giving it a duration in seconds. It will treat it like a timestamp, and give you a very low date and time.
print scalar localtime 648;
# gives Thu Jan 1 01:10:48 1970
Your code gives the following output:
Unix time 648 converts to Jan 1, 1970 1:10:48
The problem is that you are essentially mixing two concepts here.
You might want to use the Benchmark module instead, which is intended for this exact purpose.
use strict;
use warning;
use Benchmark;
my $t0 = Benchmark->new;
# do your database stuff here
my $t1 = Benchmark->new;
my $td = timediff($t1, $t0);
print "the code took:",timestr($td),"\n";
If you are on a Linux or Unix, you can also use the time program to meassure the overall runtime of your program.
$ time perl foo.pl
real 0m2.241s
user 0m2.236s
sys 0m0.000s
$
I need to calculate the time difference between these two variables in Perl :
my $end = "17:23:31,576";
my $start = "17:23:30,858";
How do I calculate the time difference ($end - $start) ? It's important the the returned value keeps the same format.
I would prefer, ideally, to do that with a native function
I've already tried packages like Time::Piece; and DateTime::Format::Strptime; but couldn't make it work.
A DateTime solution, since it has native support for fractional seconds. Once you got the two DateTime objects, you can use the following program:
# delta_ms loses nanoseconds, but we need it to convert days into hours.
my ($h, $m, $s) = $dt2->delta_ms($dt1)->in_units(qw( hours minutes seconds ));
my $ns = ( $dt2 - $dt1 )->nanoseconds;
say sprintf '%d:%02d:%02d,%03.0f', $h, $m, $s, $ns/1_000_000;
Now, let's look at how to get the two DateTime objects. Given the information provided, the best you can do is the following:
use DateTime::Format::Strptime qw( );
my $format = DateTime::Format::Strptime->new(
pattern => '%H:%M:%S,%3N',
time_zone => 'floating',
on_error => 'croak',
);
my $dt1 = $format->parse_datetime('01:59:58,123');
my $dt2 = $format->parse_datetime('03:01:02,456');
Like all other solutions, this won't always give the right answer because the difference can depend on the date and time zone (e.g. due to DST changes), and this information wasn't made available.
If you do have the information available, then you'd use the following, which correctly gives 1:01:04,333 instead of 0:01:04,333 on the day DST starts.
use DateTime::Format::Strptime qw( );
my $format = DateTime::Format::Strptime->new(
pattern => '%Y-%m-%d %H:%M:%S,%3N',
time_zone => 'America/New_York', # Often: 'local'
on_error => 'croak',
);
my $dt1 = $format->parse_datetime('2015-03-09 01:59:58,000');
my $dt2 = $format->parse_datetime('2015-03-09 03:01:02,000');
With Time::Piece, I think you have to handle the milliseconds separately.
use Time::Piece;
my $start = "17:23:31,576";
my $end = "17:23:30,858";
my ($start_time,$start_ms) = split /,/, $start;
my ($end_time,$end_ms) = split /,/, $end;
my $difference = Time::Piece->strptime($end_time,'%H:%M:%S') - Time::Piece->strptime($start_time,'%H:%M:%S') + ( $end_ms - $start_ms ) / 1000;
To turn the fractional number of seconds back into a formatted time is easy, if messy:
my $negative = $difference < 0 ? '-' : '';
$difference = abs $difference;
my $seconds = int $difference;
my $ms = sprintf '%03d', 1000 * ($difference - $seconds);
my $formatted_difference = Time::Piece->gmtime($seconds)->strftime("$negative%H:%M:%S,$ms");
Just a similar, but more manual approach, caclulating the diff manually, and using gmtime to split it out into hours, minutes and seconds. Handling fractions manually, as gmtime only uses integers.
This is kind of reinventing Time::Piece, so I actually prefer ysth's solution.
my $start = "17:23:30,858";
my $end = "17:23:31,576";
#split by ; and ,
my ($h_start,$m_start,$s_start, $frac_start) = split(/[:,]/, $start) ;
my ($h_end,$m_end,$s_end, $frac_end) = split(/[:,]/, $end) ;
#find #seconds since epoch (whatever it is)
my $start_time = $s_start + $m_start * 60 + $h_start * 3600;
my $end_time = $s_end + $m_end * 60 + $h_end * 3600;
my $diff_sec = $end_time - $start_time;
#handling fractions independently, paying attention to negative fractions
#0.0 + first to force float
my $diff_frac = 0.0 + "0.$frac_end" - "0.$frac_start";
if ($diff_frac < 0) {
$diff_frac++;
$diff_sec--;
}
#then using the build in function to convert from seconds from epoch to sec min hrs
#using gmt (not local) to convert without timezones and daylight savings...
my ($s_diff,$m_diff,$h_diff) = gmtime( $diff_sec);
#and finally manual formatting using sprintf
my $diff_date = sprintf("%02u:%02u:%02u,%03u", $h_diff,$m_diff,$s_diff,$diff_frac*1000);
print $diff_date;
How to convert date format YYYY-MM-DDTHH:MM:SSZ to YYYY-MM-DD HH:MM + 8 hours?
For example:
Input: 2011-07-07T18:05:45Z
Output: 2011-07-08 02:05
Let's start with Rahul's snippet, and add in the date math and output formatting...
use DateTime;
use DateTime::Format::ISO8601;
use DateTime::Format::Strptime;
my $string = '2011-07-07T18:05:45Z';
my $dt = DateTime::Format::ISO8601->parse_datetime( $string );
die "Impossible time" unless $dt;
my $formatter = new DateTime::Format::Strptime(pattern => '%Y-%m-%d %T');
$dt->add( hours => 8 )->set_formatter($formatter);
print "$dt\n";
I've added the use of DateTime::Format::Strptime, in order to specify the desired output format.
Then I've added three more lines:
First I create a formatter, and feed it the output pattern I desire.
Next I add eight hours to the original date, and I assign the output
formatter by chaining the set_formatter() call to the add() call.
Then I print it.
Are you using the DateTime modules?
Specifically, here's a link to DateTime::Format::ISO8601 that reads/writes ISO 8601 format you mentioned as your input.
If you don't have DateTime, you surely have Time::Piece:
use strict;
use warnings;
use Time::Piece;
use Time::Seconds qw(ONE_HOUR);
my $str = '2011-07-07T18:05:45Z';
my $t = Time::Piece->strptime($str, "%Y-%m-%dT%TZ");
$t += 8 * ONE_HOUR;
print $t->strftime("%Y-%m-%d %H:%M"),"\n";
Taken From
How can I validate a "yyyy-MM-dd'T'HH:mm:ssZ" date/timestamp in UTC with Perl?
use DateTime;
use DateTime::Format::ISO8601;
my $string = '2010-02-28T15:21:33Z';
my $dt = DateTime::Format::ISO8601->parse_datetime( $string ); die "Impossible time" unless $dt;
It doesn't work, result is 2010-02-28T15:21:33
Then, do it the hard way...
use Time::Local
use warnings;
use strict;
$time = '2010-02-28T15:21:33Z';
my ($year, month, day) = split (/-/, $time)
$year -= 1900; #Year is an offset of 1900
$month -= 1; #Months are 0 - 11
#Now split the time off of the day (DDTHH:MM:SS)
$day = substr($day, 0, 2);
time = substr($day, 3)
#Now split the time
(my $hour, $minute, $second) = split(/:/, $time);
$second =~ s/Z$//; #Remove Z
my $time_converted = timelocal($second, $minute, $hour, $day, $month, $year);
#Now you have the time, Add eight hours
my $hours_in_seconds = 8 * 60 * 60;
$time_converted += $hours_in_seconds;
# Almost done: Convert time back into the correct array:
($second, $minute, $hour, $day, $month, $year) = localtime($time_converted);
$year += 1900;
$month += 1;
# Now, reformat:
my $formatted_time = sprint (%04d-%02d-%02d %02d:%02d),
$year, $month, $day, $hour, $minute;
I have an array with n strings in format of YYYY-MM-DD (Example, "2010-10-31").
How do I compare a date to the strings in this array?
For example, delete the strings more than 30 day ago?
The great thing about YYYY-MM-DD-formatted dates is that you can compare them using simple string comparison. In Perl, that's the lt and gt operators.
In this case, it sounds like you're just looking to check whether the dates in the array are earlier or later than a given target date (which just happens to be "30 days ago"). For that case, the approach I would take would be to first determine what the date was 30 days ago and then compare that as a string against each date in the array. I would not introduce the overhead of converting all the YYYY-MM-DD strings into "proper" date objects, epoch times, etc. and back just for the sake of testing which represents the earlier date.
#!/usr/bin/env perl
use strict;
use warnings;
my $thirty_days = 30 * 24 * 60 * 60;
my ($old_day, $old_month, $old_year) = (localtime(time - $thirty_days))[3..5];
my $cutoff = sprintf('%04d-%02d-%02d',
$old_year + 1900, $old_month + 1, $old_day);
my #dates = ('2010-10-12', '2010-09-12', '2010-08-12', '2010-09-13');
for my $date (#dates) {
print "$date\n" if $date gt $cutoff;
}
Guess there's more than one way to do it, but I like Date::Simple for stuff like this ..
An example from the docs:
use Date::Simple ('date', 'today');
# Difference in days between two dates:
$diff = date('2001-08-27') - date('1977-10-05');
# Offset $n days from now:
$date = today() + $n;
print "$date\n"; # uses ISO 8601 format (YYYY-MM-DD)
It's great for doing arithmetic on objects ++.
Only dates however, no hours, minutes or seconds
use strict; use warnings;
use DateTime ();
use DateTime::Duration ();
use DateTime::Format::Natural ();
my $parser = DateTime::Format::Natural->new;
my $now = DateTime->now;
my $delta = DateTime::Duration->new( days => 30 );
my $cutoff = $now->subtract_duration( $delta );
my #new_dates = map { $_->[1] }
grep { -1 == $_->[0] }
map {
chomp;
[
DateTime->compare(
$parser->parse_datetime( $_ ),
$cutoff
),
$_
]
} <DATA>;
print "#new_dates";
__DATA__
2010-07-31
2010-08-31
2010-09-30
2010-10-31
A good start is to read The Many Dates of Perl and the DateTime site.
The YYYY-MM-DD format is a form of ISO 8601 date representation. There are variants of it that are considered acceptable, such as YYYY-MM-DD and YYYYMMDD and even YYMM in older data. You should look at a definitive list before you choose a method to compare these dates.
If ISO 8601 dates strings are: 1) valid dates; 2) in the same format with or without the - delimiter; 3) lacking in leading and trailing whitespace, an attractive property is that you can sort or compare the strings with simple lexicographical string comparisons.
In general then:
IFF you aren't going to check if the dates are valid and IFF they are the same format, and IFF there is not leading or trailing whitespace, you can compare against another string representing the target date in that same format.
--- Otherwise ---
Decide on a CPAN module to parse your date string (or match it yourself),
Convert to epoch time if if your dates are in that range, (or use a CPAN module that does larger scale date / time manipulation like Date::Manip or Date::Calc)
Perform the arithmetic on the type of time (epoch time, absolute days, whatever)
Convert the time back into the format that you want...
Here is code that does that:
use warnings; use strict;
use Date::Calc qw/:all/;
my (#date_strings, #abs_days);
my $target=Date_to_Days(2010, 1, 15);
# set #date_string to "YYYY-MM-DAY" between some dates
for my $AbsDay(Date_to_Days(2009,1,1)..Date_to_Days(2011,12,31)) {
my ($year, $mon, $day)=Add_Delta_Days(1,1,1,$AbsDay-1);
my $s="$year-$mon-$day";
push #date_strings, $s;
}
foreach my $s (#date_strings) {
my ($year, $mon, $day);
if(($year, $mon, $day)=$s=~/^(\d+)-(\d+)-(\d+)/) {
my $days=Date_to_Days($year, $mon, $day);
push #abs_days, $days
if ($target-$days <= 30 && $target-$days >= -30 );
}
}
print "absolute day way:\n";
foreach my $days (#abs_days) {
my ($year, $mon, $day)=Add_Delta_Days(1,1,1,$days-1);
print "$year-$mon-$day\n";
}
You can use Time::ParseDate module,
use strict;
use warning;
use Time::ParseDate;
my #dates = ('2010-10-12', '2010-09-14', '2010-08-12', '2010-09-13');
my #dates =
grep {parsedate($_, NO_RELATIVE => 1, UK => 1) > parsedate('-30 days') }#dates;
#output: 2010-10-12 2010-09-14
I did it like this, kind of verbose but it's easy to understand and gets the job done. #out2 is a 2d array, I'm reading in values using a for loop. Each loop I compare the input with the #out2 to see if it's an earlier or later time/date. If it is then I write the values to the array and then compare the next input.
if ($year < $out2[$j][7]) {
$lt = 1;
goto JUMP;
}
if ($year > $out2[$j][7]) {
$gt = 1;
goto JUMP;
}
if ($month < $out2[$j][5]) {
$lt = 1;
goto JUMP;
}
if ($month > $out2[$j][5]) {
$gt = 1;
goto JUMP;
}
if ($day < $out2[$j][6]) {
$lt = 1;
goto JUMP;
}
if ($day > $out2[$j][6]) {
$gt = 1;
goto JUMP;
}
if ($time < $out2[$j][4]) {
$lt = 1;
goto JUMP;
}
if ($time > $out2[$j][4]) {
$gt = 1;
goto JUMP;
}
JUMP:
if ($lt == 1) {
$out2[$j][2] = "$dtime $month\/$day\/$year";
$out2[$j][4] = $time;
$out2[$j][5] = $month;
$out2[$j][6] = $day;
$out2[$j][7] = $year;
$lt = 0;
}
if ($gt == 1) {
$out2[$j][3] = "$dtime $month\/$day\/$year";
$out2[$j][4] = $time;
$out2[$j][5] = $month;
$out2[$j][6] = $day;
$out2[$j][7] = $year;
$gt = 0;
}
Why not the CORE since 5.10 Time::Piece and Time::Seconds, not the first few results of a CPAN search?
use strict;
use warnings;
use Time::Piece (); # we don't need to include overloaded locatime
use Time::Seconds;
use Data::Dumper;
my #dates = qw/2010-10-31 2012-10-16 2011-09-08/;
my $now = Time::Piece::localtime();
my #date_objects = map {
Time::Piece->strptime( $_, '%F') # %F is the same as %Y-%m-%d
} #dates;
my #filtered_dates = grep {
$now - $_ < (ONE_DAY * 30)
} #date_objects;
print Dumper(map($_->strftime('%F'), #filtered_dates));
To find a minimal date in a loop:
var minDate = ...;
var maxDate = ...;
foreach my $date ( #$dates ) {
if ($minDate gt $date){ # Less than.
$minDate = $date; # Minimal date.
}
if ($minDate lt $date){ # Greater than.
$minDate = $date; # Maxamal date.
}
}