Convert date in YYYY-MM-DD in Perl? - perl

I have search the internet for some time now but I can't seem to find the right answer (maybe there is but I don't understand it).
I have this code to read a file and get the date time (from Task Scheduler query).
File.txt holds the Task Scheduler query.
"TaskName","Next Run Time","Status"
"CheckFile","20:33:00, 17/1/2013",""
=======================================
Script to read a get the next run time value.
open (FILE, "<", $file) || print "WARN: Cannot open $file: $!";
#logLines = <FILE>;
if (#list = grep(/\b$keyword\b/, #logLines)) {
foreach(#list){$result = $_;}
my #sresult = split(/(?<="),(?=")/, $result);
$name = $sresult[0];
$name =~ tr/"//d;
$next_run = $sresult[1];
$next_run =~ tr/"//d;
print $next_run;
}
#list=();
#dFormat=();
#logLines=();
close FILE;
Output will be:
20:33:00, 17/1/2013
I want to modify the output into:
20:33:00, 2013-1-17 #note that I can do this just by splitting up and rearranging the numbers.
But the problem is, 17/1/2013 in Task Scheduler query is locale dependent. It could be in the following:
1/17/2013
17/1/2013
2013/1/17
1/17/13
17/1/13
13/1/17
1-17-2013
17-1-2013
2013-1-17
1-17-13
17-1-13
13-1-17
1.17.2013
17.1.2013
2013.1.17
1.17.13
17.1.13
13.1.17
Is there any cpan module that could do what I want? Could you give a script on how to achieve this?
Please no harsh comment. Thanks.

The following should get you the format:
use Win32::OLE::NLS qw( GetLocaleInfo GetSystemDefaultLCID LOCALE_SSHORTDATE );
say GetLocaleInfo(GetSystemDefaultLCID(), LOCALE_SSHORTDATE); # yyyy-MM-dd
You could try to find a date parser that understands that format, or you could use something like the following to create a format many parsers to understand.
my %subs = (
'yyyy' => '%Y',
...
);
my $pat = join '|', map quotemeta, sort { length($b) <=> length($a) } keys %subs;
$format =~ s{($pat)}{ $subs{$1} // $1 }eg;

I can tell you answer(algorithm) about your problem:
Once upon a time
Let's think about your problem:
You want to get date from every format.
But this is not possible, because it can be 12-11-10
Okay, now you must determine (somehow) which format is used.
If you can do that, you can rule the world.
Let's think about your problem more deeper:
First you must choose delimiter. (or not, if you don't want it)
it is possible to use something like this:
(\d{1,4})(\D)(\d{1,4})(\D)(\d{1,4})
After that, you have:
$1,$3,$5 # parts of data
$2,$4 # delimiters
I took 4 because i don't know where year can be placed.
After that, you must understand, which format you have:
dd-mm-yy
mm-dd-yy
yy-mm-dd
yy-dd-mm
You can add checks for that like days or months, but, as I said before:
12-11-10 = -9 <- not possible to determine, right?
So, you must have some external info about date format, for example:
which country
which branch of science
which family
etc
it belongs to.
If you can do that, you can (probably can) determine format and 12-11-10 = 12 nov 2010
The End

Update: see this discussion. It appears that the date format may be a regional setting based on the user who runs the task. If so, all you have to do is figure out how to get that regional setting...
As others have pointed out, there is no way of resolving the ambiguities in the potential date formats.
What I would explore is some way of querying the system with a known date and see what format it returns. For example, perhaps you could schedule a fake task at a (non-ambiguous) date far in the future, see what format that task is returned in, then later delete it.
That would be a bit of a messy solution, but perhaps there is a less kludgy way of doing something similar. Is task scheduler's date format the same as the system date format? If so, you could query a known date from the system.

Related

logstash reformat dates at OUTPUT time

I'm outputting to CSV and I'd like my dates in ISO8601 format, such as 2014-04-02T19:21:36.292Z, but I keep getting dates like Mar 27 2014 17:56:33 in my csv.
I'm fine to create a second intermediate string variable to do the formatting, but it yields the same result.
I see that there's a "sprintf" function in Logstash, but it seems you can do EITHER variable references OR date formats (which I assume will get the current system date time), but I don't think you can do both. I other words, I don't think it lets you apply a date format to an existing date variable, or if it does I'm not sure what the syntax would be.
Plenty of false hits on Google and stack, but all are about parsing.
Ironically stdout happens to output in the format I want, using stdout { debug => true codec => "rubydebug"}. Maybe that could somehow help in my case, not sure? Although other folks might want some other arbitrary format.
Try this one. Add a new "date" field then output to csv.
filter {
ruby {
code => '
require "time"
event["date"] = Time.parse(event["#timestamp"].to_s).iso8601;
'
}
}

Determining Local Time in another Timezone

How can I determine the current date and time of various countries using a PERL script that executes on a server in the US? For example, getDTnow() should determine the current date and time on the server and use that to return the date and time of various countries.
P.S: It would be great if this can be done using only the built-in functions, without any external modules.
Conclusion: Date maths is [use swear word here] complicated and easy to get wrong. Other perl gurus on IRC, groups and other parts of the net confirmed what Ether had been advicing me - use DateTime. DVK's solution is also pretty neat for those of you who don't mind messing with the perl environment. (Note: Though on windows, the caveats section of the Time::Piece docs says one should be careful while 'Setting $ENV{TZ} in Threads on Win32').
DateTime is a wonderful library that can use standard timezones to do everything you desire and more:
use DateTime;
# returns local time in Italy
my $dt = DateTime->now(time_zone => 'Europe/Rome');
# prints time in desired format
print "The current date and time in Italy is: ", $dt->strftime('%Y-%m-%d %T');
You can control which timezone localtime returns in via TZ environmental variable:
local $ENV{TZ} = ":/usr/share/lib/zoneinfo/Asia/Tokyo";
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday) = localtime();
print "$sec,$min,$hour,$mday,$mon,$year,$wday,$yday\n"'
# Prints 40,58,4,12,0,111,3,11
local $ENV{TZ} = ":/usr/share/lib/zoneinfo/Europe/London";
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday) = localtime();
print "$sec,$min,$hour,$mday,$mon,$year,$wday,$yday\n"'
# Prints 41,58,19,11,0,111,2,10
Unfortunately, the path above is different on different Unixes (/usr/share/lib/zoneinfo on Solaris, /usr/share/zoneinfo on Linux). Since there appear to be no other variations, a slightly portable version would check which of the 2 directories exists and use that - but this obviously only works on Solaris and Linux and may be other unixes. No idea about Windows/MacOS/whatnot.
Valid locations for TZ can be found here: http://www.timezoneconverter.com/cgi-bin/tzref.tzc (but not all of them would necessarily be available on your system - check the above directory).
Please see http://en.wikipedia.org/wiki/Tz_database for more info on TZ database.
You could always store the variation from your timezone in a hash where the key is the timezone and the value is the adjustment from the current time. then when you pass the current time it should return the local time for that zone.

How do I parse this file and store it in a table?

I have to parse a file and store it in a table. I was asked to use a hash to implement this. Give me simple means to do that, only in Perl.
-----------------------------------------------------------------------
L1234| Archana20 | 2010-02-12 17:41:01 -0700 (Mon, 19 Apr 2010) | 1 line
PD:21534 / lserve<->Progress good
------------------------------------------------------------------------
L1235 | Archana20 | 2010-04-12 12:54:41 -0700 (Fri, 16 Apr 2010) | 1 line
PD:21534 / Module<->Dir,requires completion
------------------------------------------------------------------------
L1236 | Archana20 | 2010-02-12 17:39:43 -0700 (Wed, 14 Apr 2010) | 1 line
PD:21534 / General Page problem fixed
------------------------------------------------------------------------
L1237 | Archana20 | 2010-03-13 07:29:53 -0700 (Tue, 13 Apr 2010) | 1 line
gTr:SLC-163 / immediate fix required
------------------------------------------------------------------------
L1238 | Archana20 | 2010-02-12 13:00:44 -0700 (Mon, 12 Apr 2010) | 1 line
PD:21534 / Loc Information Page
------------------------------------------------------------------------
I want to read this file and I want to perform a split or whatever to extract the following fields in a table:
the id that starts with L should be the first field in a table
Archana20 must be in the second field
timestamp must be in the third field
PD must be in the fourth field
Type (content preceding / must be in the last field)
My questions are:
How to ignore the --------… (separator line) in this file?
How to extract the above?
How to split since the file has two delimiters (|, /)?
How to implement it using a hash and what is the need for this?
Please provide some simple means so that I can understand since I am a beginner to Perl.
My questions are:
How to ignore the --------… (separator line) in this file?
How to extract the above?
How to split since the file has two delimiters (|, /)?
How to implement it using a hash and what is the need for this?
You will probably be working through the file line by line in a loop. Take a look at perldoc -f next. You can use regular expressions or a simpler match in this case, to make sure that you only skip appropriate lines.
You need to split first and then handle each field as needed after, I would guess.
Split on the primary delimiter (which appears to be ' | ' - more on that in a minute), then split the final field on its secondary delimiter afterwards.
I'm not sure if you are asking whether you need a hash or not. If so, you need to pick which item will provide the best set of (unique) keys. We can't do that for you since we don't know your data, but the first field (at a glance) looks about right. As for how to get something like this into a more complex data structure, you will want to look at perldoc perldsc eventually, though it might only confuse you right now.
One other thing, your data above looks like it has a semi-important typo in the first line. In that line only, there is no space between the first field and its delimiter. Everywhere else it's ' | '. I mention this only because it can matter for split. I nearly edited this, but maybe the data itself is irregular, though I doubt it.
I don't know how much of a beginner you are to Perl, but if you are completely new to it, you should think about a book (online tutorials vary widely and many are terribly out of date). A reasonably good introductory book is freely available online: Beginning Perl. Another good option is Learning Perl and Intermediate Perl (they really go together).
When you say This is not a homework...to mean this will be a start to assess me in perl I assume you mean that this is perhaps the first assignment you have at a new job or something, in which case It seems that if we just give you the answer it will actually harm you later since they will assume you know more about Perl than you do.
However, I will point you in the right direction.
A. Don't use split, use regular expressions. You can learn about them by googling "perl regex"
B. Google "perl hash" to learn about perl hashes. The first result is very good.
Now to your questions:
regular expressions will help you ignore lines you don't want
regular expressions with extract items. Look up "capture variables"
Don't split, use regex
See point B above.
If this file is line based then you can do a line by line based read in a while loop. Then skip those lines that aren't formatted how you wish.
After that, you can either use regex as indicated in the other answer. I'd use that to split it up and get an array and build a hash of lists for the record. Either after that (or before) clean up each record by trimming whitespace etc. If you use regex, then use the capture expressions to add to your list in that fashion. Its up to you.
The hash key is the first column, the list contains everything else. If you are just doing a direct insert, you can get away with a list of lists and just put everything in that instead.
The key for the hash would allow you to look at particular records for fast lookup. But if you don't need that, then an array would be fine.
You can try this one,
Points need to know:
read the file line by line
By using regular expression, removing '----' lines.
after that use split function to populate Hashes of array .
#!/usr/bin/perl
use strict;
use warning;
my $test_file = 'test.txt';
open(IN, '<' ,"$test_file") or die $!;
my (%seen, $id, $name, $timestamp, $PD, $type);
while(<IN>){
chomp;
my $line = $_;
if($line =~ m/^-/){ #removing '---' lines
# print "$line:hello\n";
}else{
if ($line =~ /\|/){
($id , $name, $timestamp) = split /\|/, $line, 4;
} else{
($PD, $type) = split /\//, $line , 3;
}
$seen{$id}= [$name, $timestamp, $PD, $type]; //use Hashes of array
}
}
for my $test(sort keys %seen){
my $test1 = $seen{$test};
print "$test:#{$test1}\n";
}
close(IN);

How can I make $1 return alternatives without a substitution regex?

The project I recently joined abstracts logic into code and database elements. Business logic like xPaths, regular expressions and function names are entered in the database, while general code like reading files, creating xml from xpaths, etc are in the code base.
Most (if not all) of the methods that use regular expressions are structured thus:
if ( $entry =~ /$regex/ ) { $req_value = $1; }
This means that only $1 is available and you always have to write your regex to give you your desired result in $1.
The issue:
The result for the following strings should be either
'2.6.9-78.1.6.ELsmp (SMP)' or '2.6.9-78.1.6.ELsmp'
depending on the existence of SMP. $1 does not suffice for $entry[0].
$entry[0] = qq|Linux version 2.6.9-78.1.6.ELsmp (brewbuilder#hs20-bc2-2.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-10)) #1 SMP Wed Sep 24 05:41:12 EDT 2008|;
$entry[1] = qq|Linux version 2.6.9-78.0.5.ELsmp (brewbuilder#hs20-bc2-2.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-10)) #1 Wed Sep 24 05:41:12 EDT 2008|;
Hence my solution:
my $mutable = '';
my $regex = qr/((\d.*?)\s+(?:.*)?(SMP)((?{$mutable="$2 ($3)"}))|(\d.*?))\s+/;
if ($entry[$i] =~ /$regex/) {
$req_value = $1;
$req_value = $mutable if ($mutable ne '');
$mutable = '';
}
Unfortunately, the existence of a 'variable' in the database makes this solution unacceptable.
My questions are:
How can I clean up the above solution to make it acceptable with the structure available?
or
How can I use a substitution regex with the structure 'if ($entry =~ /$regex/)'?
Thanks.
You're stuck unless you can talk the folks who control the code you're using into generalizing it somehow. The good news is you need only a bit more, perhaps
if (my #fields = $_ =~ /$pat/) {
$req_value = join " " => grep defined($_), #fields;
}
This works because a successful regular-expression match in list context returns all captured substrings, i.e., $1, $2, $3, and so on as appropriate.
With a single pattern,
qr/(\d+(?:[-.]\w+)*)(?:.*(SMP))?/
the code above yields 2.6.9-78.1.6.ELsmp SMP and 2.6.9-78.0.5.ELsmp in $req_value. The grep defined($_) filters out captures for subpatterns not taken. Without it, you get undefined value warnings for the non-SMP case.
The downside is every regular expression would need to be reviewed to be sure that all capturing groups really ought to go in $req_value. For example, say someone is using the pattern
qr/(XYZ) OS (version \d+|v-\d+)/
As it is now, only XYZ would go into $req_value, but using the above generalization would also include the version number. If that's undesired, the regular expression should be
qr/(XYZ) OS (?:version \d+|v-\d+)/
because (?:...) does not capture (that is, it does not produce a $2 for the pattern above): it's for grouping only.
I don't fully understand your constraints. Are you limited to supplying a single regex that will always by processed using the code in your first excerpt? If so, you cannot do what you are trying to do. You are trying to extract two separate parts of the entry string, you simply can't return 2 values in a single scalar return value unless you can add the code to concatenate them.
Can you add perl code at all? For example, can you define the logic to be:
if ( $entry =~ /$regex/ ) { $req_value = '$1 $2'; }
where your $regex = qr/((\d.*?)\s+(?:.*)?(SMP)/; ?
Baring the ability to define some new perl code, you can't accomplish this.
Regarding part two, substiutions. I interpret your question to ask if you can compile both the PATTERN and REPLACEMENT parts of s/PATTERN/REPLACEMENT/ into a single qr//. If so, you cannot. qr// only compiles a matching pattern, and a qr variable can only be used in the PATTERN portion of a REPLACEMENT. In other words, to use s///, you'll need to write perl code that runs s///. I'm guessing that if you could write new perl code, you'd use the above solution.
One more thought: In your current architecture, can you define fields in terms of of other fields? In other words, could you extract the version string with one regex, the SMP string with another regex, and define a third field that combines the two?
As of 5.10.0, (?|pattern) is available to allow alternatives to use the same capture numbering. As you pointed out that you're still using 5.8, this may not be useful directly but perhaps as further incentive to your project to start moving to a modern Perl.

How I can check for a valid date format in Perl?

I am getting a date field from the database in one of my variables, at the moment I am using the following code to check if the date is in "yyyy-mm-dd" format
if ( $dat =~ /\d{3,}-\d\d-\d\d/ )
My question, is there a better way to accomplish this.
Many Thanks
The OWASP Validation Regex Repository's version of dates in US format with support for leap years:
^(?:(?:(?:0?[13578]|1[02])(/|-|.)31)\1|(?:(?:0?[1,3-9]|1[0-2])(/|-|.)(?:29|30)\2))(?:(?:1[6-9]|[2-9]\d)?\d{2})$|^(?:0?2(/|-|.)29\3(?:(?:(?:1[6-9]|[2-9]\d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:(?:0?[1-9])|(?:1[0-2]))(/|-|.)(?:0?[1-9]|1\d|2[0-8])\4(?:(?:1[6-9]|[2-9]\d)?\d{2})$
The Regular Expression Library contains a simpler version along the lines of the other suggestions, which is translated to your problem:
^\d{4}-\d{1,2}-\d{1,2}$
As noted by others, if this is a date field from a database, it should be coming in a well-defined format, so you can use a simple regex, such as that given by toolkit.
But that has the disadvantage that it will accept invalid dates, such as 2009-02-30. Again, if you're handling dates that successfully made it into a date-typed field in a DB, you should be safe.
A more robust approach would be to use one of the many Date/Time modules from CPAN. Probably Date::Manip would be a good choice, and in particular check out the ParseDate() function.
http://metacpan.org/pod/Date::Manip
How about
/\d{2}\d{2}?-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[0-1])/
\d could match number characters from other languages. And is YYY really a valid year? If it must be four digits, dash, two digits, dash, two digits, I'd prefer /^[0-9]{4}-[0-9]{2}-[0-9]{2}$/ or /^[12][0-9]{3}-[0-9]{2}-[0-9]{2}$/. Be aware of space characters around the string you're matching.
Of course, this doesn't check the reasonableness of the characters that are there, except for the first character in the second example. If that's required, you'll do well to just pass it to a date parsing module and then check its output for logical results.
The best and lightweight solution is using Date::Calc's check_date sub routine, here's an example:
use strict;
use warnings
use Date::Calc qw[check_date];
## string in YYYY-MM-DD format, you can have any format
## you like, just parse it
my #dt_dob = unpack("A4xA2xA2",$str_dob_date);
unless(check_date(#dt_dob)) {
warn "Oops! invalid date!";
}
I hope that was helpful :-)
Well you can start with:
/\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|30|31)/
I would very strongly recommend AGAINST writing your own regular expression to do this. Date/time parsing is simple, but there are some tricky aspects, and this is a problem that has been solved hundreds of times. No need for you to design, write, and debug yet another solution.
If you want a regular expression, the best solution is probably to use my Regexp::Common::time plugin for the Regexp::Common module. You can specify simple or complex, rigid or fuzzy date/time matching, and it has a very extensive test suite.
If you just want to parse specific date formats, you may be better off using one of the many parsing/formatting plugins for Dave Rolsky's excellent DateTime module.
If you want to validate the date/time values after you have matched them, I would suggest my Time::Normalize module.
Hope this helps.
I think using a regex without outer check is much to complicated! I use a little sub to get it:
sub check_date {
my $date_string = shift;
# Check the string fromat and get year, month and day out of it.
# Best to use a regex.
return 0 unless $date_string =~ m/^(\d{4})-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01])$/;
# 31. in a month with 30 days
return 0 if ($3 >= 31 and ($2 == 4 or $2 == 6 or $2 == 9 or $2 == 11));
# February 30. or 31.
return 0 if ($3 >= 30 and $2 == 2);
# February 29. in not a leap year.
return 0 if ($2 == 2 and $3 == 29
and not ($1 % 4 == 0 and ($1 % 100 != 0 or $1 % 400 == 0)));
# Date is valid
return 1;
}
I got the idea (and most of the code) from regular-expressions.info. There are other examples too.