How can I get array of Sundays between two dates? - perl

I'm starting out with two dates.
my $date1 = 01/01/2016;
my $date2 = 05/15/2016;
I need to put all the dates of the Sundays between two dates.
Any idea where I should start?

Your solution is good, but it potentially consumes a lot of memory creating an array of Sundays that might never be used. DateTime objects are not small nor cheap.
An alternative is an iterator, a function which every time it's called generates the next Sunday. It generates each Sunday on demand rather than calculating them all beforehand. This saves memory and supports potentially infinite Sundays.
use strict;
use warnings;
use v5.10;
sub sundays_iterator {
my($start, $end) = #_;
# Since we're going to modify it, copy it.
$start = $start->clone;
# Move to Sunday, if necessary.
$start->add( days => 7 - $start->day_of_week );
# Create an iterator using a closure.
# This will remember the values of $start and $end as
# they were when the function was returned.
return sub {
# Clone the date to return it without modifications.
# We always start on a Sunday.
my $date = $start->clone;
# Move to the next Sunday.
# Do this after cloning because we always start on Sunday.
$start->add( days => 7 );
# Return Sundays until we're past the end date
return $date <= $endDate ? $date : ();
};
}
That returns a closure, an anonymous subroutine which remembers the lexical variables it was created with. Sort of like an inside out object, a function with data attached. You can then call it like any subroutine reference.
my $sundays = sundays_iterator($startDate, $endDate);
while( my $sunday = $sundays->() ) {
say $sunday;
}
The upside is it saves a lot of memory, this can be especially important if you're taking the dates as user input: a malicious attacker can ask you for an enormous range consuming a lot of your server's memory.
It also allows you to separate generating the list from using the list. Now you have a generic way of generating Sundays within a date range (or, with a slight tweak, any day of the week).
The downside is it's likely to be a bit slower than building an array in a loop... but probably not noticeably so. Function calls are relatively slow in Perl, so making one function call for each Sunday will be slower than looping, but calling those DateTime methods (which call other methods which call other methods) will swamp that cost. Compared to using DateTime, calling the iterator function is a drop in the bucket.

You should start by picking a module. I'm partial to DateTime, using DateTime::Format::Strptime for the parsing.
use DateTime qw( );
use DateTime::Format::Strptime qw( );
my $start_date = "01/01/2016";
my $end_date = "05/15/2016";
my $format = DateTime::Format::Strptime->new(
pattern => '%m/%d/%Y',
time_zone => 'floating', # Best not to use "local" for dates with no times.
on_error => 'croak',
);
my $start_dt = $format->parse_datetime($start_date)->set_formatter($format);
my $end_dt = $format->parse_datetime($end_date )->set_formatter($format);
my $sunday_dt = $start_dt->clone->add( days => 7 - $start_dt->day_of_week );
while ($sunday_dt <= $end_dt) {
print "$sunday_dt\n";
$sunday_dt->add( days => 7 );
}
Note: You really shouldn't use DateTime->new as Bill used and Schwern endorsed. It's not the recommended use of DateTime because it creates code that's far more complicated and error-prone. As you can see, using a formatter cut the code size in half.
Note: Schwern is advocating the use of an iterator, replacing the last four lines of my answer with something 4 times longer (all the code in his answer). There's no reason for that high level complexity! He goes into length saying how much memory the iterator is saving, but it doesn't save any at all.

DateTime::Set makes constructing an iterator easy:
use DateTime::Format::Strptime ();
use DateTime::Set ();
my $start_date = "01/01/2016";
my $end_date = "05/15/2016";
my $format = DateTime::Format::Strptime->new(
pattern => '%m/%d/%Y',
time_zone => 'local',
on_error => 'croak',
);
my $iterator = DateTime::Set->from_recurrence(
start => $format->parse_datetime($start_date)->set_formatter($format),
end => $format->parse_datetime($end_date)->set_formatter($format),
recurrence => sub { $_[0]->add( days => 7 - $_[0]->day_of_week || 7 ) }, # next Sunday after $_[0]
);
while ( my $date = $iterator->next ) {
say $date;
}

This is what I came up with but please let me know if there is a better way.
use DateTime;
my $date1 = "1/1/2016";
my $date2 = "5/15/2016";
my ($startMonth, $startDay, $startYear) = split(/\//, $date1);
my ($endMonth, $endDay, $endYear) = split(/\//, $date2);
my $startDate = DateTime->new(
year => $startYear,
month => $startMonth,
day => $startDay
);
my $endDate = DateTime->new(
year => $endYear,
month => $endMonth,
day => $endDay
);
my #sundays;
do {
my $date = DateTime->new(
year => $startDate->year,
month => $startDate->month,
day => $startDate->day
);
push #sundays, $date if ($date->day_of_week == 7);
$startDate->add(days => 1);
} while ($startDate <= $endDate);
foreach my $sunday (#sundays) {
print $sunday->strftime("%m/%d/%Y");
}

Related

Perl variable scopes seemingly not cooperating

All my variables are declared following my $var format.
In my program I call mainMethod()
Then in mainMethod I define a DateTime variable and then pass it to a separate helper method 3 times.:
my ($day,$month,$year)=(localtime)[3,4,5];
my $dt = DateTime->new(
year => $year,
month => 2 + $month,
day => 1,
hour => 8,
minute => 30,
time_zone => 'America/New_York'
);
helper($dt, $year, $month, $day,);
helper($dt,$year, $month, $day,);
helper($dt,$year, $month, $day,);
In helper I start with:
my $date = $_[0];
my ($year, $month, $day) = ($_[1],$_[2],$_[3]);
This method does a lot of different things but the only changes I make to $date is $date = $date->add(days => (3 - $date->day_of_week) % 7);
The problem is, it seems like the changes I make $date in my helper method, persist through each time I call the method from mainMethod Note: I'm not returning anything
For example, if $dt was instead 5 and helper was $date + 2 I would get
helper($dt) #output: 7
helper($dt) #output: 9
helper($dt) #output: 11
Instead of the expected 5 for every response.
Is there some step that I'm missing to reset those variables at the end of the method?
DateTime creates a reference.
So when you call helper($dt) you pass a copy of the reference to helper. It doesn't create a whole new DateTime object.
Any changes you make to the value you pass are made to the single DateTime object you have already created.
If you want to create three different DateTime objects, then you need to call DateTime->new three times.
This has nothing to do with scoping. Your DateTime object is a reference. That's how objects work in Perl.
To explain what's going on, I will use a simpler data structure. An array reference works just the same way.
my $foo = [];
helper($foo);
print "#$foo\n";
helper($foo);
print "#$foo\n";
helper($foo);
print "#$foo\n";
sub helper {
my $bar = $_[0];
push #$bar, 1;
}
The output of this program will be
1
1 1
1 1 1
Just like with your DateTime object, we're passing a reference to the helper() function. Passing a scalar value like this creates a copy, but because the variable only contains a reference to somewhere else in memory, it copies that pointer. It will still point at the same place in memory, so any changes you make to that will be reflected in your original DateTime object.

Is there any point to this code that translates a strftime into a time and then back into a string?

I am converting some scripts from Perl to Python. I have a piece of code that looks weird.
my $dt = DateTime->now->subtract( days => 1 );
my $start_time = $dt->strftime("%Y-%m-%d");
# ... <some code here>
my $epoch = str2time( $start_time, "GMT" );
my $start_dt = DateTime->from_epoch( epoch => $epoch );
my $start_date = $start_dt->strftime("%Y-%m-%d");
And here I have a question about start_time and start_date.
Does it make sense to perform these last three lines?
UPDATE:
#darch asked to add any useful code related to start_time. I don't think it is useful, but here it is (removed not significant lines):
my %opts;
my $opt_string = 'ahropuc:i:s:e:t:';
getopts( "$opt_string", \%opts );
my $start_time = $dt->strftime("%Y-%m-%d");
if ( $opts{a} ) {
if ( $opts{s} ) {
$start_time = $opts{s};
}
}
my $epoch = str2time( $start_time, "GMT" );
my $start_dt = DateTime->from_epoch( epoch => $epoch );
my $start_date = $start_dt->strftime("%Y-%m-%d");
The only reasonable answer is validation as answered below.
I am going to rewrite this part to exclude user input, so if it is and this part is meaningless - I simply cut it.
Normalization. Those lines are providing normalization of user input.
You see, Date::Parse::str2time can accept input in a wide variety of formats . By parsing the user input and then turning it back into a string, we can guarantee that we use a consistent format internally.
The existing code can be restructured into this equivalent formulation:
my %opts;
my $opt_string = 'ahropuc:i:s:e:t:';
getopts( "$opt_string", \%opts );
my $start_time;
# use the time specified on the command line or 'yesterday'
if ( $opts{a} and $opts{s} ) {
my $start_time_string = $opts{s};
my $epoch = Date::Parse::str2time( $start_time_string, "GMT" );
$start_time = DateTime->from_epoch( epoch => $epoch );
} else {
$start_time = DateTime->now->subtract( days => 1 );
}
# should be called $start_date_string, but keep the old name
# so that we could just drop this code into place
my $start_date = $start_time->strftime("%Y-%m-%d");
If you are not worried about accepting user input, you should be able to safely remove the lines that perform the normalization.

Running into a memory error using Perl and DateTime

I am writing a small tool to parse some application logs for to collect data that is going to be used as the inputs for Zabbix monitoring. I am just wanting to keep data from the logs that are within the past two hours.
The format of the logs is pretty simple, the fields are separated by white space and the first three fields are used to determine the time when the logging was written.
Here is an example of the first three fields of a log line:
Jan 5 13:42:07
What I set out to do was to utilize one of my favorite modules, DateTime. Where I convert the above into a DateTime object and then compare that object to another DateTime object when the utility would be invoked.
Everything was fine an dandy and working nicely until I actually set the utility against the a portion of the logs it would actually be parsing -- only a couple gigabytes in size. The test run was being done on a kitchen invoked Ubuntu virtual box instance on my laptop, so the resources are -- as expected -- rather limited. The script would halt with the words 'Killed' displayed.
Looking into /var/log/messages I would see log lines describing the process being killed due to resource issues.
When I invoked the process again, and then switching to another screen instance to watch top, I noticed that the memory percentage would grow, that swap space would being to be consumed all until the script would again stop with the 'Killed' message.
When I would rerun the script with the DateTime portion commented out, the script would execute as expected.
In the script I have a subroutine which would be called to create a DateTime object based upon the information found in the first three fields of the log line. I have tried where I create the object at the beginning of the subroutine then undef it prior to returning a value at the end of the subroutine, I have tried it where I create a global object ( using our ) and then use the DateTime set_* methods to modify what I thought would be a single object's values.
I have read that perl does not clean up hash memory so that it can be reused by the program--I feel that this is the base of the issue that I am running into.
At this point, I am feel the need to get input of others and that is the reason for this post. All comments and criticisms would be appreciated.
This utility was running on Perl v5.14.2.
This code produces the memory leak:
#!/usr/bin/perl -w
use strict;
use DateTime;
my $month = 1;
my $day = 6;
my $hour = 20;
my $minute = 30;
my $second = 00;
for (my $count = 0; $count <= 25_000_000; $count++) {
my $epoch = &get_epoch( $month, $day, $hour, $minute, $second );
}
sub get_epoch {
my $mon = shift;
my $day = shift;
my $hour = shift;
my $min = shift;
my $sec = shift;
my $temp_dt = DateTime->new(
year => 2015,
month => $mon,
day => $day,
hour => $hour,
minute => $min,
second => $sec,
nanosecond => 500_000_000,
time_zone => 'UTC',
);
return( $temp_dt->epoch );
}
This is a bug in Params::Validate 1.15 and will be fixed very soon.

How to make DateTime::Duration output only in days?

This code finds the difference between today and a fixed date.
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
use DateTime ();
use DateTime::Duration ();
use DateTime::Format::Strptime ();
my $date = "23/05-2022";
my $parser = DateTime::Format::Strptime->new(
pattern => '%d/%m-%Y',
time_zone => 'local',
);
$date = $parser->parse_datetime($date);
my $today = DateTime->today(time_zone=>'local');
my $d = DateTime::Duration->new($today - $date);
print Dumper $d->delta_days;
The problem is that is only outputs -22 days.
If I do print Dumper $d; I can see the -130 months as well.
$VAR1 = bless( {
'seconds' => 0,
'minutes' => 0,
'end_of_month' => 'preserve',
'nanoseconds' => 0,
'days' => -22,
'months' => -130
}, 'DateTime::Duration' );
How do I get it to output the result in days?
Doing
print Dumper $d->delta_days + $d->delta_months*30;
doesn't seam like an elegant solution.
At first you need to do the correct subtraction. There exists delta_md, delta_days, delta_ms and subtract_datetime_absolute. Depending on which unit you later want, you need to pick the right subtraction. The problem is that not every unit is convertible later without time_zone information. Thats the reason why you need to pick the correct delta method.
For example a day can have 23 Hours or 24 or 25 Hours, depending on the time zone. Because of that, you need to specify how the subtraction should work. Because you want the days later, the subtraction need to focus on days, rather focus on hours. Don't use the overload feature, because it only does a best fit.
That means you need to do a delta_days subtraction.
my $dur = $date->delta_days($today);
Now $dur is a DateTime::Duration object. You need to knew that it always tries to best fit the days, weeks, years, months if possible. That means your days will split in weeks and days. Because this conversion is always a constant.
If you don't want this "best fit" you need to call the in_units method and convert it only to days.
my $days = $dur->in_units('days');
But like i said before in_units only can do a conversion where it is possible. A call with in_units('hours') will not work on this object and just return a zero because you cant convert days to hours. If you want hours for example, you need to do a delta_ms, and on this object you can call in_units('hours')
The complete example:
#!/usr/bin/env perl
use 5.010;
use strict;
use warnings;
use DateTime;
use DateTime::Format::Strptime;
my $date = "23/05-2022";
my $parser = DateTime::Format::Strptime->new(
pattern => '%d/%m-%Y',
time_zone => 'local',
);
$date = $parser->parse_datetime($date);
my $today = DateTime->new(
day => 1,
month => 7,
year => 2011,
time_zone => 'local'
);
my $dur = $date->delta_days($today);
say "Weeks: ", $dur->weeks;
say "Days: ", $dur->days;
say "Absolute Days: ", $dur->in_units('days');
say "Absolute Hours: ", $date->delta_ms($today)->in_units('hours');
The output of this program is:
Weeks: 568
Days: 3
Absolute Days: 3979
Absolute Hours: 95496
And just for info:
1) You don't need to load DateTime::Duration its get loaded with DateTime.
2) You dont need (). These modules are OOP and don't export/import anything.
From a quick read of the DateTime module doc, I don't believe that
DateTime::Duration->new($today - $date)
will do what you expect. I believe you need to use
$dur = $today->subtract_datetime($date)
The type of $dur is not immediately clear from the docs, however.

How can I test Perl applications using a changing system time?

I have a web application that I want to run some system tests on, and in order to do that I'm going to need to move the system time around. The application used DateTime all the way through.
Has anyone got any recommendations for how to change the time that DateTime->now reports? The only thing that comes to mind is subclassing DateTime and messing about with all the 'use' lines, but this seems rather invasive.
Note on answers:
All three will work fine, but the Hook::LexWrap one is the one I've chosen because (a) I want to move the clock rather than jiggle it a bit (which is more the purpose of what Time::Mock and friends do); (b) I do, consistently, use DateTime, and I'm happy to have errors come out if I've accidentally not used it; and (c) Hook::LexWrap is simply more elegant than a hack in the symbol table, for all that it does the same thing. (Also, it turns out to be a dependency of some module I already installed, so I didn't even have to CPAN it...)
Rather than taking the high-level approach and wrapping DateTime specifically, you might want to look into the modules Test::MockTime and Time::Mock, which override the low-level functions that DateTime etc. make use of, and (with any luck) will do the right thing on any time-sensitive code. To me it seems like a more robust way to test.
I think Hook::LexWrap is overkill for this situation. It's easier to just redefine such a simple function.
use DateTime;
my $offset;
BEGIN {
$offset = 24 * 60 * 60; # Pretend it's tomorrow
no warnings 'redefine';
sub DateTime::now
{
shift->from_epoch( epoch => ($offset + scalar time), #_ )
}
} # end BEGIN
You can replace my $offset with our $offset if you need to access the $offset from outside the file which contains this code.
You can adjust $offset at any time, if you want to change DateTime's idea of the current time during the run.
The calculation of $offset should probably be more complicated than shown above. For example, to set the "current time" to an absolute time:
my $want = DateTime->new(
year => 2009,
month => 9,
day => 14,
hour => 12,
minute => 0,
second => 0,
time_zone => 'America/Chicago',
);
my $current = DateTime->from_epoch(epoch => scalar time);
$offset = $want->subtract_datetime_absolute($current)->in_units('seconds');
But you probably do want to calculate a fixed number of seconds to add to the current time, so that time will advance normally after that. The problem with using add( days => 1 ); in the redefined now method is that things like DST changes will cause the time to jump at the wrong pseudotime.
You can use code injection via Hook::LexWrap to intercept the now() method.
use Hook::LexWrap;
use DateTime;
# Use real now
test();
{
my $wrapper = wrap 'DateTime::now',
post => sub {
$_[-1] = DateTime->from_epoch( epoch => 0 );
};
# Use fake now
test();
}
# use real now again
test();
sub test {
my $now = DateTime->now;
print "The time is $now\n";
}
When designing a new class with testability in mind, the ideal solution is to be able to inject new date objects.
However, for existing code using DateTime->now and DateTime->today a possible, suitably scoped, solution is below. I include it here as a way to do this without introducing Hook::LexWrap as a dependency and without affecting the behaviour globally.
{
no strict 'refs';
no warnings 'redefine';
local *{'DateTime::today'} = sub {
return DateTime->new(
year => 2012,
month => 5,
day => 31
);
};
say DateTime->today->ymd(); # 2012-05-31
};
say DateTime->today->ymd(); # today