my variable inside the properties file is $starttime and the value is current date in YYYYMMDDHH24MI after i run second time a new variable with $stattime_2 with current date value.
my code is
#!/usr/local/bin/perl
use Time::Piece;
$starttime = localtime->strftime('%Y%m%d%H%M');
$i = 0;
open my $file, '>', 'order.properties' or die $!;
print $file "Start_time", $i, " = ", $starttime;
close $file;
for each run the order.properties file should update like
at first time
Start_time_1 = 2018121317:04(the current system Time)
at second time
Start_time_2 = 2018121317:05.........
3rd,4th,5th the variable name should change and current date and time should assign
OUTPUT will be like
at 3rd run
Start_time_1 = 2018121317:04
Start_time_2 = 2018121317:05
Start_time_3 = 2018121317:09
How may execution of Script equal to the entries of start time in the properties file
I'm not going to give you a complete answer as you'll learn more by working it out for yourself. But I will point out the two things you'll need to fix.
You open your file using >, which overwrites the file each time you run your program. You need to, instead, use "append" mode, which adds new data to the end of your file. You do that by using >> instead of >.
You also need to work out which number gets appended to Start_time. Obviously, your program closes down each time it finishes, so you can't store it as a variable. I would suggest that the easiest approach is probably to count the lines that are currently in the file before writing your new lines.
Two more pieces of advice. The Perl FAQ is a great source of Perl programming advice and you should always have use strict and use warnings in your Perl programs.
Related
Was debugging a perl script for the first time in my life and came over this:
$my_temp_file = File::Temp->tmpnam();
system("cmd $blah | cmd2 > $my_temp_file");
open(FIL, "$my_temp_file");
...
unlink $my_temp_file;
This works pretty much like I want, except the obvious race conditions in lines 1-3. Even if using proper tempfile() there is no way (I can think of) to ensure that the file streamed to at line 2 is the same opened at line 3. One solution might be pipes, but the errors during cmd might occur late because of limited pipe buffering, and that would complicate my error handling (I think).
How do I:
Write all output from cmd $blah | cmd2 into a tempfile opened file handle?
Read the output without re-opening the file (risking race condition)?
You can open a pipe to a command and read its contents directly with no intermediate file:
open my $fh, '-|', 'cmd', $blah;
while( <$fh> ) {
...
}
With short output, backticks might do the job, although in this case you have to be more careful to scrub the inputs so they aren't misinterpreted by the shell:
my $output = `cmd $blah`;
There are various modules on CPAN that handle this sort of thing, too.
Some comments on temporary files
The comments mentioned race conditions, so I thought I'd write a few things for those wondering what people are talking about.
In the original code, Andreas uses File::Temp, a module from the Perl Standard Library. However, they use the tmpnam POSIX-like call, which has this caveat in the docs:
Implementations of mktemp(), tmpnam(), and tempnam() are provided, but should be used with caution since they return only a filename that was valid when function was called, so cannot guarantee that the file will not exist by the time the caller opens the filename.
This is discouraged and was removed for Perl v5.22's POSIX.
That is, you get back the name of a file that does not exist yet. After you get the name, you don't know if that filename was made by another program. And, that unlink later can cause problems for one of the programs.
The "race condition" comes in when two programs that probably don't know about each other try to do the same thing as roughly the same time. Your program tries to make a temporary file named "foo", and so does some other program. They both might see at the same time that a file named "foo" does not exist, then try to create it. They both might succeed, and as they both write to it, they might interleave or overwrite the other's output. Then, one of those programs think it is done and calls unlink. Now the other program wonders what happened.
In the malicious exploit case, some bad actor knows a temporary file will show up, so it recognizes a new file and gets in there to read or write data.
But this can also happen within the same program. Two or more versions of the same program run at the same time and try to do the same thing. With randomized filenames, it is probably exceedingly rare that two running programs will choose the same name at the same time. However, we don't care how rare something is; we care how devastating the consequences are should it happen. And, rare is much more frequent than never.
File::Temp
Knowing all that, File::Temp handles the details of ensuring that you get a filehandle:
my( $fh, $name ) = File::Temp->tempfile;
This uses a default template to create the name. When the filehandle goes out of scope, File::Temp also cleans up the mess.
{
my( $fh, $name ) = File::Temp->tempfile;
print $fh ...;
...;
} # file cleaned up
Some systems might automatically clean up temp files, although I haven't care about that in years. Typically is was a batch thing (say once a week).
I often go one step further by giving my temporary filenames a template, where the Xs are literal characters the module recognizes and fills in with randomized characters:
my( $name, $fh ) = File::Temp->tempfile(
sprintf "$0-%d-XXXXXX", time );
I'm often doing this while I'm developing things so I can watch the program make the files (and in which order) and see what's in them. In production I probably want to obscure the source program name ($0) and the time; I don't want to make it easier to guess who's making which file.
A scratchpad
I can also open a temporary file with open by not giving it a filename. This is useful when you want to collect outside the program. Opening it read-write means you can output some stuff then move around that file (we show a fixed-length record example in Learning Perl):
open(my $tmp, "+>", undef) or die ...
print $tmp "Some stuff\n";
seek $tmp, 0, 0;
my $line = <$tmp>;
File::Temp opens the temp file in O_RDWR mode so all you have to do is use that one file handle for both reading and writing, even from external programs. The returned file handle is overloaded so that it stringifies to the temp file name so you can pass that to the external program. If that is dangerous for your purpose you can get the fileno() and redirect to /dev/fd/<fileno> instead.
All you have to do is mind your seeks and tells. :-) Just remember to always set autoflush!
use File::Temp;
use Data::Dump;
$fh = File::Temp->new;
$fh->autoflush;
system "ls /tmp/*.txt >> $fh" and die $!;
#lines = <$fh>;
printf "%s\n\n", Data::Dump::pp(\#lines);
print $fh "How now brown cow\n";
seek $fh, 0, 0 or die $!;
#lines2 = <$fh>;
printf "%s\n", Data::Dump::pp(\#lines2);
Which prints
[
"/tmp/cpan_htmlconvert_DPzx.txt\n",
"/tmp/cpan_htmlconvert_DunL.txt\n",
"/tmp/cpan_install_HfUe.txt\n",
"/tmp/cpan_install_XbD6.txt\n",
"/tmp/cpan_install_yzs9.txt\n",
]
[
"/tmp/cpan_htmlconvert_DPzx.txt\n",
"/tmp/cpan_htmlconvert_DunL.txt\n",
"/tmp/cpan_install_HfUe.txt\n",
"/tmp/cpan_install_XbD6.txt\n",
"/tmp/cpan_install_yzs9.txt\n",
"How now brown cow\n",
]
HTH
As part of a pipeline I am trying to create I need to take an output from a process that is a .tab results file to use as a variable for running a series of further conditional processes in Nextflow. I can manage this in Perl with something like below, although I can't seem to add the variable to the Nextflow script:
my $resultsfile = 'path/to/resultsfile';
open (RESLIST, "<$resultsfile") or die$!;
my #resultlist=<RESLIST>;
$LINE = $resultlist[1];
close RESLIST or die$!;
my #topresults = split ("\t", $LINE);
my $topidentity = $topresults[5];
chomp $topidentity
Is there a way to take the variable $topidentity to use as a variable in Nextflow or perhaps a different way to run a similar script in Nextflow to produce the same variable?
One groovy oneliner should do that, for example:
file('your-file.tab').text.readLines()[1].tokenize('\t')[5]
(didn't test it and quickly adapted from your perl script. it may need more adaptation)
My problem is simple one...
My perl code is like below...
$todaydate = `date +%Y-%m-%d-%H%M%S`;
$output_file = "my_data_$todaydate".".csv";
print "SQL query output file name : $output_file\n";
But the Output file name is printing as like below...
SQL query output file name : my_data_2017-10-03-062227
.csv
If you can observe, the .csv is coming in new line.
I have also tried the below join for string conactantion. but still no luck.
$output_file = join "", "my_data_", $todaydate, ".csv";
due to this issue, while i am passing the output_file name to a sql query, its creating a file my_data_2017-10-03-062227 without .csv extension.
Any suggestion please...
There are several reasons why you might not want to use external programs unnecessarily.
The external program might not be available (or might work differently) on some systems where you want to run your program. You are therefore making your code less portable.
Starting a new shell and invoking an external program takes longer than just using a Perl feature to achieve the same result.
The value returned from an external problem will probably have a newline character at the end - and you might forget to remove it.
Getting a date is a task that people commonly want to use an external program for. And I don't understand why, because Perl has pretty good built-in time and date handling. For example, your code can be written like this:
use Time::Piece;
$todaydate = localtime->strftime('%Y-%m-%d-%H%M%S');
$output_file = "my_data_$todaydate.csv";
print "SQL query output file name : $output_file\n";
Time::Piece has been included with all versions of Perl since 2007. It changes the behaviour of localtime() so it returns an object. And the object has many useful methods - here we use strftime().
If you're stuck with an older version of Perl (pre-5.10) then you can still do this easily without calling an external program.
use POSIX 'strftime';
$todaydate = strftime('%Y-%m-%d-%H%M%S', localtime);
$output_file = "my_data_$todaydate.csv";
print "SQL query output file name : $output_file\n";
use chomp in your $todaydate variable.
my $todaydate = `date +%Y-%m-%d-%H%M%S`;
chomp $todaydate;
my $output_file = "my_data_$todaydate.csv";
Always put use warnings; and use strict; in top of the program.
Its because of new line character at the end of $todaydate. Use below to fix it:
$todaydate =~ s/\n|\r//g;
The above code will remove any occurrence of \n or \r.
I am connecting SFTP and downloading the file using perl. I want to download the file that is created/modified 1 hours back.
The below is code snippet.
use strict;
use Net::SFTP::Foreign;
my $sftp_conn=Net::SFTP::Foreign->new('test.sftp.com',user=>'test',password=>'test123');
my $a1 = Net::SFTP::Foreign::Attributes->new();
my $a2 = $sftp_conn->stat('/inbox/tested.txt') or die "remote stat command failed: ".$sftp_conn->status;
$sftp_conn->get("/inbox/tested.txt","/tmp"));
Here I want to check the file age at what time it modified and calculate in hours.
You are on the right track. Calling ->stat on the connection object returns a Net::SFTP::Foreign::Attributes object. You can then call ->mtime on it to get the modifcation time.
my $attr = $sftp_conn->stat('/inbox/tested.txt')
or die "remote stat command failed: ".$sftp_conn->status;
print $attr->mtime;
There is no need to create an empty object first. You don't need the following line. You probably copied it from the SYNOPSIS in the docs, but that's just to show different ways of using that module. You can delete it.
my $a1 = Net::SFTP::Foreign::Attributes->new();
I don't know which format the mtime will be in, so I can't tell you how to do the comparison. There is nothing about that in the docs, in the code of the module or in the tests.
A quick google suggested "YYYYMMDDhhmmss", but that might not be the right one. Just try it. If it's a unix timestamp, you can just compare it to time or time - 3600, but if it's a string, you will need to parse it. Time::Piece is a useful module that comes with the Perl core to do that.
Been away from Perl awhile and would like to modify a script I wrote as an art project long ago. The original script uses Term::ReadKey to allow the user to type arbitrary text into a Mac/Linux terminal. As they type, the text creates various floaty patterns in the terminal. I want to adapt the script so, instead of reading keys as they're input, it can read from text files written periodically by another process. But it would need to read in characters in some controllable way (not all at once) so as to (roughly) emulate human typing.
What I've tried:
Term::ReadKey's man page says it can read from a filehandle instead of STDIN - but for some reason, I could not get this to work, with either a standard file or a FIFO. I also tried reading in text from a file using "open" and putting the characters into an array. But iterating through the array got complicated because of the need to add delays between characters without pausing the rest of the script. ( I can envision this as a potential solution, but I'm not sure how best to design it to allow time delays to be controllable without the script becoming unwieldy.)
Wondering if there's a relatively simple way to approach this - assuming it's feasible at all?
Here's the "meat" of the existing script (have removed various subroutines that add additional effects based on various keypresses.)
#!/usr/bin/perl
use Time::HiRes(usleep);
use Term::ReadKey;
$|=1;
$starttime = time;
$startphrase = ' ';
$startsleepval = 3000;
$phrase = $startphrase;
$sleepval = $startsleepval;
$dosleep = 1;
$SIG{'INT'}=\&quitsub;
$SIG{'QUIT'}=\&quitsub;
# One Ctrl-C clears text and resets program. # Three Ctrl-C's to quit.
sub quitsub {print color 'reset' if ($dosleep); $phrase = $startphrase; $sleepval=$startsleepval; $SIG{'INT'}=\&secondhit;}
sub secondhit { $SIG{'INT'}=\&outtahere; }
sub outtahere {print color 'reset'; sleep 1; print "\n\n\t\t\t\n\n"; exit(0);}
while (1) {
print "$phrase ";
if ($dosleep) {
usleep ($sleepval);
}
ReadMode 3;
##### Here is where it reads from the terminal. Can characters be read from a file in a similar sequential fashion? #####
$key = ReadKey(-1);
$now = time;
if ((defined($key)) and ($now > $starttime + 5)) {
$phrase = $phrase.$key;
$SIG{'INT'}=\&quitsub;
}
# user can also create interesting effects with spacebar, tab and arrow keys.
ReadMode 0; # this may appear redundant, but has a subtle visual effect. At least that's what I commented in the original 2003 script.
}
# end main loop
The problem here is that your script can try to read from the file all you want, but if the process that actually writes to the file flushes all at once, you're going to get everything together.
Also, a few things:
if you really want to use ReadKey, you should probably use ReadMode 5 if you don't know the CR or CR/LF use of your file.
also check Term::ReadKey and you'll see that you probably want something like ReadKey 0, $file
it's probably best if you drop Term::ReadKey completely and use File::Tail instead, looping over the added characters one at a time
Your final code is most likely going to be something that goes through an array of characters, just as you've already tried to do.