perl POSIX::strptime - my data looks like 9:31:00 AM - I want to extract the minute and perform a calculation - perl

I want to extract the minute from a specified time that exists in the 2nd column of a comma delimited file and perform a calculation. The format of the time is as follows:
9:31:00 AM
I want to extract the minute value and calculate the total minutes in the day so far. I do this in subroutine get_time. But the returned value is always zero which makes me think that I am not using POSIX::strptime correctly. Any insight would be wonderful. Thank you.
#!/usr/bin/env perl
use strict;
use POSIX::strptime;
use Text::CSV;
sub get_time
{
my($str) = #_;
my ($sec, $min, $hour) = (POSIX::strptime($str, '%I:%M:%S')) [3,4,5];
print "$hour\n";
return($hour*60 + $min)
}
open my $fh, "<", datafile.txt
my $csv = Text::CSV->new() or die "Failed to create Text::CSV object";
my $line = <$fh>;
die "Failed to parse line <<$line>>\n\n" unless $csv->parse($line);
my #columns = $csv->fields();
while ($line = <$fh>)
{
chomp $line;
die "Failed to parse line <<$line>>\n\n" unless $csv->parse($line);
my #columns = $csv->fields();
die "Insufficient columns in <<$line>>\n" if scalar(#columns) < 1;
my $minute = get_time($columns[1]);
print "$minute\n";
}

In get_time, you have the line
my ($sec, $min, $hour) = (POSIX::strptime($str, '%I:%M:%S')) [3,4,5];`
According to the docs, strptime returns
($sec, $min, $hour, $mday, $mon, $year, $wday, $yday) = POSIX::strptime("string", "Format");
So it looks like you need
my ($sec, $min, $hour) = (POSIX::strptime($str, '%I:%M:%S')) [0,1,2];
instead.
Best of luck!

It's impossible to correctly determining the number of a minutes in a day before a certain time without knowing the time zone and possibly the date.
$ perl -MDateTime -E'
for (10..16) {
my $dt = DateTime->new(
year => 2011,
month => 3,
day => $_,
hour => 9,
minute => 31,
second => 0,
time_zone => "America/New_York",
);
( my $ref_dt = $dt->clone )->truncate(to => "day");
my $minutes = ($dt - $ref_dt)->in_units("minutes");
say($dt->ymd, " ", $minutes);
}
'
2011-03-10 571
2011-03-11 571
2011-03-12 571
2011-03-13 511 <------
2011-03-14 571
2011-03-15 571
2011-03-16 571

Related

Perl Time, date conversion Error

This is a perl script for sql data pulling each day for 100 days starting from Oct 1 and
SQL is quite picky in date formats(yyyy-mm-dd), so I've written the script as follows.
However, at a specific day, on 2011-11-06, the time to date conversion is incorrect, and start and end date become the same.
$srt_date='2011-11-06'
$end_date='2011-11-06'
I don't know if this is perl error or something else.
use DBI;
use DBD::Oracle qw(:ora_types);
use Compress::Zlib;
use FileHandle;
use Date::Parse;
use Date::Format;
$st_day=str2time('2011-10-1');
#days=(0..100);
foreach $daynum (#days){
$dt1 = $st_day+3600*(24*$daynum);
$dt2 = $st_day+3600*(24*($daynum+1));
$srt_date = time2str("%d-%h-%Y", $dt1);
$end_date = time2str("%d-%h-%Y", $dt2);
print $srt_date, ',' ,$end_date, '\n';
my $sqlGetEid = "select x,y from z where DATETIME>='$srt_date' and DATETIME<'$end_date'";
}
Here's how DateTime handles the DST transitions correctly:
use strict; #ALWAYS!
use warnings; #ALWAYS!
use DateTime;
my $st_day = '2011-10-1';
my ($year, $month, $day) = split /-/, $st_day;
my $dt = DateTime->new(
year => $year,
month => $month,
day => $day,
time_zone => 'local',
);
my #days = 0..100;
foreach my $daynum (#days) {
my $dt1 = $dt->ymd;
my $dt2 = $dt->add(days => 1)->ymd;
printf "%s,%s\n", $dt1, $dt2;
}
I'm not sure what you want to achieve exactly, but why bother executing 100 SQL statements when you can get away with something like:
SELECT trunc(datetime, 'DD') truncdate, x,y
FROM z WHERE datetime between '2011-10-01'
AND to_date('20111001', 'YYYYMMDD') + 99
Populate a hash with truncdate as key, and if your dates are ISO 8601, you'll get the same ordering by looping over the hash with a regular (cmp) sort.
EDIT: I'll clarify how you could do this:
my $sth = $mdbh->prepare("SELECT trunc(datetime, 'DD') truncdate, x,y
FROM z WHERE datetime between '2011-10-01'
AND to_date('20111001', 'YYYYMMDD') + 99
ORDER BY truncdate");
$sth->execute();
my $lastdate = "";
my $fh;
while (my $row = $sth->fetchrow_hashref()) {
# If new date, create new file
if ($row->{truncdate} ne $lastdate) {
close($fh) if $fh;
open($fh, ">", "$row->{truncdate}.csv") or die "Unable to create file '$row->{truncdate}.csv': $!\n";
}
print $fh "$row->{x},$row->{y}\n";
$lastdate = $row->{truncdate};
}
close($fh) if $fh;

Convert Old Unix Date to Perl and compare

Requirement - I have file name called "Rajesh.1202242219". Numbers are nothing but a date "date '+%y''%m''%d''%H''%M'" format.
Now I am trying to write a perl script to extract the numbers from file name and compare with current system date and time and based on output of this comparison, print some value using perl.
Approach:
Extract the Digit from File name:
if ($file =~ /Rajesh.(\d+).*/) {
print $1;
}
Convert this time into readable time in perl
my $sec = 0; # Not Feeded
my $min = 19;
my $hour = 22;
my $day = 24;
my $mon = 02 - 1;
my $year = 2012 - 1900;
my $wday = 0; # Not Feeded
my $yday = 0; # Not Feeded
my $unixtime = mktime ($sec, $min, $hour, $day, $mon, $year, $wday, $yday);
print "$unixtime\n";
my $readable_time = localtime($unixtime);
print "$readable_time\n";
find Current time and compare...
my $CurrentTime = time();
my $Todaydate = localtime($startTime);
But the problem here is, I am not getting solution of how to extract 2 digit from $1 and assign to $sec, $min, etc. Any help?
Also, if you have good approach for this problem statement, Please share with me
I like to use time objects to simplify the logic. I use Time::Piece here because it is simple and light weight (and part of the core). DateTime can be another choice.
use Time::Piece;
my ( $datetime ) = $file =~ /(\d+)/;
my $t1 = Time::Piece->strptime( $datetime, '%y%m%d%H%M' );
my $t2 = localtime(); # equivalent to Time::Piece->new
# you can do date comparisons on the object
if ($t1 < $t2) {
# do something
print "[$t1] < [$t2]\n";
}
Might as well teach DateTime::Format::Strptime to make the comparison much simpler:
use DateTime qw();
use DateTime::Format::Strptime qw();
if (
DateTime::Format::Strptime
->new(pattern => '%y%m%d%H%M')
->parse_datetime('Rajesh.1202242219')
< DateTime->now
) {
say 'filename timestamp is earlier than now';
} else {
say 'filename timestamp is later than now';
};
my ($year, $month, $day, $hour, $min) = $file =~ /(\d{2})/g;
if ($min) {
$year += 100; # Assuming 2012 and not 1912
$month--;
# Do stuff
}
I think unpack might be a better fit.
if ( my ( $num ) = $file =~ /Rajesh.(\d+).*/ ) {
my ( $year, $mon, $day, $hour, $min ) = unpack( 'A2 A2 A2 A2 A2', $num );
my $ts = POSIX::mktime( 0, $min, $hour, $day, $mon - 1, $year + 100 );
...
}
Using a module that parses dates might be nice. This code will parse the date and return a DateTime object. Refer to the documentation to see the many ways to manipulate this object.
use DateTime::Format::Strptime;
my $date = "1202242219";
my $dt = get_obj($date);
sub get_obj {
my $date = shift;
my $strp = DateTime::Format::Strptime->new(
pattern => '%y%m%d%H%M'
);
return $strp->parse_datetime($date);
}

In Perl, how can I ensure that a string corresponds to a valid date?

I was wondering if there is a simple way in Perl to ensure that a date string corresponds to a valid date.
For example, 2012 02 30 is incorrect because it doesn't exist.
The DateTime module will validate dates when creating a new object.
$ perl -we 'use DateTime; my $dt;
eval { $dt = DateTime->new(
year => 2012,
month => 2,
day => 30);
}; print "Error: $#" if $#;'
Error: Invalid day of month (day = 30 - month = 2 - year = 2012) at -e line 1
It also works dynamically on a given DateTime object:
$dt->set(day => 30);
Something like this using Class::Date should work
perl testit.pl
Range check on date or time failed
use Class::Date;
my $d=Class::Date->new('2021-02-30');
unless ( $d->error ) {
print "good date\n";
} else {
print $d->errstr(). "\n";
}
exit;
Check here:
http://www.perlmonks.org/?node_id=564594
I believe you'll get the answers you seek from the wise monks.
You can do this through the use of POSIX mktime, but apparently only if you have a flexible-enough implementation of mktime.
What I do is plug the numbers in and then use local time to get them back and if I get the same day value back, it's a valid number. So, given your string:
my ( $y, $m, $d ) = split ' ', $date_string;
die "$date_string is not a valid date!"
unless ( $d == ( localtime mktime( 0, 0, 0, $d, $m - 1, $y - 1900 ))[3] )
;
See, in the versions of mktime that I'm used to, mktime( 0, 0, 0, 30, 1, 112 ) would make '2012-03-01' and 30 != 1
You can also use Time::Local:
#!/usr/bin/env perl
use strict; use warnings;
use Carp qw( croak );
use Time::Local qw( timegm );
my #to_check = ('1927 06 18', '2012 02 30');
for my $date ( #to_check ) {
printf "'%s' is %s\n", $date, check_date($date) ? 'valid' : 'invalid';
}
sub check_date {
my ($date) = #_;
my ($year, $month, $mday) = split ' ', $date;
my $ret;
eval {
$ret = timegm(0, 0, 0, $mday, $month - 1, $year - 1900);
};
return $ret && $ret;
}
May be this will help too:
use Time::Piece; #in perl CORE distro since 5.10
use 5.010;
say Time::Piece->strptime("2011-02-29","%Y-%m-%d")->strftime("%Y-%m-%d");
#2011-03-01
say Time::Piece->strptime("2012-02-29","%Y-%m-%d")->strftime("%Y-%m-%d");
#2012-02-29

Perl - How to convert a date?

How to convert date format YYYY-MM-DDTHH:MM:SSZ to YYYY-MM-DD HH:MM + 8 hours?
For example:
Input: 2011-07-07T18:05:45Z
Output: 2011-07-08 02:05
Let's start with Rahul's snippet, and add in the date math and output formatting...
use DateTime;
use DateTime::Format::ISO8601;
use DateTime::Format::Strptime;
my $string = '2011-07-07T18:05:45Z';
my $dt = DateTime::Format::ISO8601->parse_datetime( $string );
die "Impossible time" unless $dt;
my $formatter = new DateTime::Format::Strptime(pattern => '%Y-%m-%d %T');
$dt->add( hours => 8 )->set_formatter($formatter);
print "$dt\n";
I've added the use of DateTime::Format::Strptime, in order to specify the desired output format.
Then I've added three more lines:
First I create a formatter, and feed it the output pattern I desire.
Next I add eight hours to the original date, and I assign the output
formatter by chaining the set_formatter() call to the add() call.
Then I print it.
Are you using the DateTime modules?
Specifically, here's a link to DateTime::Format::ISO8601 that reads/writes ISO 8601 format you mentioned as your input.
If you don't have DateTime, you surely have Time::Piece:
use strict;
use warnings;
use Time::Piece;
use Time::Seconds qw(ONE_HOUR);
my $str = '2011-07-07T18:05:45Z';
my $t = Time::Piece->strptime($str, "%Y-%m-%dT%TZ");
$t += 8 * ONE_HOUR;
print $t->strftime("%Y-%m-%d %H:%M"),"\n";
Taken From
How can I validate a "yyyy-MM-dd'T'HH:mm:ssZ" date/timestamp in UTC with Perl?
use DateTime;
use DateTime::Format::ISO8601;
my $string = '2010-02-28T15:21:33Z';
my $dt = DateTime::Format::ISO8601->parse_datetime( $string ); die "Impossible time" unless $dt;
It doesn't work, result is 2010-02-28T15:21:33
Then, do it the hard way...
use Time::Local
use warnings;
use strict;
$time = '2010-02-28T15:21:33Z';
my ($year, month, day) = split (/-/, $time)
$year -= 1900; #Year is an offset of 1900
$month -= 1; #Months are 0 - 11
#Now split the time off of the day (DDTHH:MM:SS)
$day = substr($day, 0, 2);
time = substr($day, 3)
#Now split the time
(my $hour, $minute, $second) = split(/:/, $time);
$second =~ s/Z$//; #Remove Z
my $time_converted = timelocal($second, $minute, $hour, $day, $month, $year);
#Now you have the time, Add eight hours
my $hours_in_seconds = 8 * 60 * 60;
$time_converted += $hours_in_seconds;
# Almost done: Convert time back into the correct array:
($second, $minute, $hour, $day, $month, $year) = localtime($time_converted);
$year += 1900;
$month += 1;
# Now, reformat:
my $formatted_time = sprint (%04d-%02d-%02d %02d:%02d),
$year, $month, $day, $hour, $minute;

How can I sum data over five minute time intervals in Perl?

I have a file in below format.
DATE Time, v1,v2,v3
05:33:25,n1,n2,n3
05:34:25,n4,n5,n5
05:35:24,n6,n7,n8
and so on upto 05:42:25.
I want calculate the values v1, v2 and v3 for every 5 min interval. I have written the below sample code.
while (<STDIN>) {
my ($dateTime, $v1, $v2, $v3) = split /,/, $_;
my ($date, $time) = split / /, $dateTime;
}
I can read all the values but need help to sum all the values for every 5 min interval. Can anyone please suggest me the code to add the time and values for every 5 min.
Required output
05:33 v1(sum 05:33 to 05:37) v2(sum 05:33 to 05:33) v3(sum 05:33 to 05:33)
05:38 v1(sum 05:38 to 05:42) v2(sum 05:38 to 05:42) v3(sum 05:38 to 05:42)
and so on..
The code is a variation the previous answer by Sinan Ünür below, except:
(1) Function timelocal will allow you to read in Day,Month,Year -- so you can sum any five minute gap.
(2) Should deal with case where final time gap is < 5 minutes.
#!/usr/bin/perl -w
use strict;
use warnings;
use Time::Local;
use POSIX qw(strftime);
my ( $start_time, $end_time, $current_time );
my ( $totV1, $totV2, $totV3 ); #totals in time bands
while (<DATA>) {
my ( $hour, $min, $sec, $v1, $v2, $v3 ) =
( $_ =~ /(\d+)\:(\d+)\:(\d+)\,(\d+),(\d+),(\d+)/ );
#convert time to epoch seconds
$current_time =
timelocal( $sec, $min, $hour, (localtime)[ 3, 4, 5 ] ); #sec,min,hr
if ( !$end_time ) {
$start_time = $current_time;
$end_time = $start_time + 5 * 60; #plus 5 min
}
if ( $current_time <= $end_time ) {
$totV1 += $v1;
$totV2 += $v2;
$totV3 += $v3;
}
else {
print strftime( "%H:%M:%S", localtime($start_time) ),
" $totV1,$totV2,$totV3\n";
$start_time = $current_time;
$end_time = $start_time + 5 * 60; #plus 5 min
( $totV1, $totV2, $totV3 ) = ( $v1, $v2, $v3 );
}
}
#Print results of final loop (if required)
if ( $current_time <= $end_time ) {
print strftime( "%H:%M:%S", localtime($start_time) ),
" $totV1,$totV2,$totV3\n";
}
__DATA__
05:33:25,29,74,96
05:34:25,41,69,95
05:35:25,24,38,55
05:36:25,96,63,70
05:37:25,84,65,74
05:38:25,78,58,93
05:39:25,51,38,19
05:40:25,86,40,64
05:41:25,80,68,65
05:42:25,4,93,81
Output:
05:33:25 352,367,483
05:39:25 221,239,229
Obviously, not tested much, for lack of sample data. For parsing the CSV, use either Text::CSV_XS or Text::xSV rather than the naive split below.
Note:
This code does not make sure the output has all consecutive five minute blocks if the input data has gaps.
You will have problems if there are time stamps from multiple days. In fact, if the time stamps are not in 24-hour format, you will have problems even if the data are from a single day.
With those caveats, it should still give you a starting point.
#!/usr/bin/perl
use strict;
use warnings;
my $split_re = qr/ ?, ?/;
my #header = split $split_re, scalar <DATA>;
my #data;
my $time_block = 0;
while ( my $data = <DATA> ) {
last unless $data =~ /\S/;
chomp $data;
my ($ts, #vals) = split $split_re, $data;
my ($hr, $min, $sec) = split /:/, $ts;
my $secs = 3600*$hr + 60*$min + $sec;
if ( $secs > $time_block + 300 ) {
$time_block = $secs;
push #data, [ $time_block ];
}
for my $i (1 .. #vals) {
$data[-1]->[$i] += $vals[$i - 1];
}
}
print join(', ', #header);
for my $row ( #data ) {
my $ts = shift #$row;
print join(', ',
sprintf('%02d:%02d', (localtime($ts))[2,1])
, #$row
), "\n";
}
__DATA__
DATE Time, v1,v2,v3
05:33:25,1,3,5
05:34:25,2,4,6
05:35:24,7,8,9
05:55:24,7,8,9
05:57:24,7,8,9
Output:
DATE Time, v1, v2, v3
05:33, 10, 15, 20
05:55, 14, 16, 18
This is a good problem for Perl to solve. The hardest part is taking the value from the datetime field and identifying which 5 minute bucket it belongs to. The rest is just hashes.
my (%v1,%v2,%v3);
while (<STDIN>) {
my ($datetime,$v1,$v2,$v3) = split /,/, $_;
my ($date,$time) = split / /, $datetime;
my $bucket = &get_bucket_for($time);
$v1{$bucket} += $v1;
$v2{$bucket} += $v2;
$v3{$bucket} += $v3;
}
foreach my $bucket (sort keys %v1) {
print "$bucket $v1{$bucket} $v2{$bucket} $v3{$bucket}\n";
}
Here's one way you could implement &get_bucket_for:
my $first_hhmm;
sub get_bucket_for {
my ($time) = #_;
my ($hh,$mm) = split /:/, $time; # looks like seconds are not important
# buckets are five minutes apart, but not necessarily at multiples of 5 min
# (i.e., buckets could go 05:33,05:38,... instead of 05:30,05:35,...)
# Use the value from the first time this function is called to decide
# what the starting point of the buckets is.
if (!defined $first_hhmm) {
$first_hhmm = $hh * 60 + $mm;
}
my $bucket_index = int(($hh * 60 + $mm - $first_hhmm) / 5);
my $bucket_start = $first_hhmm + 5 * $bucket_index;
return sprintf "%02d:%02d", $bucket_start / 60, $bucket_start % 60;
}
I'm not sure why you would use the times starting from the first time, instead of round 5 minute intervals (00 - 05, 05 - 10, etc), but this is a quick and dirty way to do it your way:
my %output;
my $last_min = -10; # -10 + 5 is less than any positive int.
while (<STDIN>) {
my ($dt, $v1, $v2, $v3) = split(/,/, $_);
my ($h, $m, $s) = split(/:/, $dt);
my $ts = $m + ($h * 60);
if (($last_min + 5) < $ts) {
$last_min = $ts;
}
$output{$last_min}{1} += $v1;
$output{$last_min}{2} += $v2;
$output{$last_min}{3} += $v3;
}
foreach my $ts (sort {$a <=> $b} keys %output) {
my $hour = int($ts / 60);
my $minute = $ts % 60;
printf("%01d:%02d v1(%i) v2(%i) v3(%i)\n", (
$hour,
$minute,
$output{$ts}{1},
$output{$ts}{2},
$output{$ts}{3},
));
}
Not sure why you would do it this way, but there you go in procedural Perl, as example. If you need more on the printf formatting, go here.