I am writing a command line php script which does some output to the console window, its all look good only issues is when i type
php myfilename.php -....
in the console window, ONLY AFTER its fully executed it outputs the result to the window ..
Wht i want is to do this on the fly like below
customer id: 1223 skipped.
customer id: 22233 added..
...etc
another question is adding \n\r to the printf functions didn't go to a new line ...
any idea on these issues..
This is probably due to output buffering. You can use ob_flush() to flush the buffer manually when needed.
As for your second issue, the correct sequence for newline on Microsoft Windows is "\r\n", not the other way around.
First, the Windows-style end-of-line marker is \r\n, not \n\r. Not many systems ever used \n\r, but they are rare enough that you can forget about them now.
Second, chances are good the output is being block buffered -- you can either use ob_implicit_flush(1) to automatically insert a flush() command after every output command. Or, you could call flush() manually, when you need to flush output.
About the End-Of-Line marker, always use PHP predefined constant PHP_EOL; it is correctly set based on your platform, so you do not have to worry is it right or wrong.
For the [Enter] issue, it could be the output buffering is on. Add this simple test in your script:
function test()
{
$state = array(' added', ' skipped');
for ($i = 0; $i < 50; $i++)
{
echo 'customer id: ' . rand(1, 1000) . $state[rand(0, 1)] . PHP_EOL;
usleep(50000); // slow it a bit to see the line by line output
}
}
// without ob -------------------------------------------
$start = microtime(true);
test();
echo 'Finished in ' . round(microtime(true) - $start, 2) . PHP_EOL . str_repeat('-', 78) . PHP_EOL;
sleep(1);
// with ob ----------------------------------------------
$start = microtime(true);
ob_start(); // if called somewhere at the top in your script
// some previous code...
echo 'Line 1'.PHP_EOL.'Line 2'.PHP_EOL.uniqid().PHP_EOL;
// flush the buffer and stop ob
// this will echo whatever is in the output buffer!
//ob_end_flush();
// or, store the current buffer content in a variable and use it later
$output = ob_get_clean();
test();
echo $output;
echo 'Finished in ' . round(microtime(true) - $start, 2) . PHP_EOL . str_repeat('-', 78) . PHP_EOL;
// you could start buffering again, if needed
ob_start();
For output control functions see http://www.php.net/manual/en/ref.outcontrol.php. They are very powerful tools.
Hope it helps. Cheers!
Related
I have a Perl CGI script for online concordance application that searches for an instance of word in a text and prints the sorted output.
#!/usr/bin/perl -wT
# middle.pl - a simple concordance
# require
use strict;
use diagnostics;
use CGI;
# ensure all fatals go to browser during debugging and set-up
# comment this BEGIN block out on production code for security
BEGIN {
$|=1;
print "Content-type: text/html\n\n";
use CGI::Carp('fatalsToBrowser');
}
# sanity check
my $q = new CGI;
my $target = $q->param("keyword");
my $radius = $q->param("span");
my $ordinal = $q->param("ord");
my $width = 2*$radius;
my $file = 'concordanceText.txt';
if ( ! $file or ! $target ) {
print "Usage: $0 <file> <target>\n";
exit;
}
# initialize
my $count = 0;
my #lines = ();
$/ = ""; # Paragraph read mode
# open the file, and process each line in it
open(FILE, " < $file") or die("Can not open $file ($!).\n");
while(<FILE>){
# re-initialize
my $extract = '';
# normalize the data
chomp;
s/\n/ /g; # Replace new lines with spaces
s/\b--\b/ -- /g; # Add spaces around dashes
# process each item if the target is found
while ( $_ =~ /\b$target\b/gi ){
# find start position
my $match = $1;
my $pos = pos;
my $start = $pos - $radius - length($match);
# extract the snippets
if ($start < 0){
$extract = substr($_, 0, $width+$start+length($match));
$extract = (" " x -$start) . $extract;
}else{
$extract = substr($_, $start, $width+length($match));
my $deficit = $width+length($match) - length($extract);
if ($deficit > 0) {
$extract .= (" " x $deficit);
}
}
# add the extracted text to the list of lines, and increment
$lines[$count] = $extract;
++$count;
}
}
sub removePunctuation {
my $string = $_[0];
$string = lc($string); # Convert to lowercase
$string =~ s/[^-a-z ]//g; # Remove non-aplhabetic characters
$string =~ s/--+/ /g; #Remove 2+ hyphens with a space
$string =~s/-//g; # Remove hyphens
$string =~ s/\s=/ /g;
return($string);
}
sub onLeft {
#USAGE: $word = onLeft($string, $radius, $ordinal);
my $left = substr($_[0], 0, $_[1]);
$left = removePunctuation($left);
my #word = split(/\s+/, $left);
return($word[-$_[2]]);
}
sub byLeftWords {
my $left_a = onLeft($a, $radius, $ordinal);
my $left_b = onLeft($b, $radius, $ordinal);
lc($left_a) cmp lc($left_b);
}
# process each line in the list of lines
print "Content-type: text/plain\n\n";
my $line_number = 0;
foreach my $x (sort byLeftWords #lines){
++$line_number;
printf "%5d",$line_number;
print " $x\n\n";
}
# done
exit;
The perl script produces expected result in terminal (command line). But the CGI script for online application produces unexpected output. I cannot figure out what mistake I am making in the CGI script. The CGI script should ideally produce the same output as the command line script. Any suggestion would be very helpful.
Command Line Output
CGI Output
The BEGIN block executes before anything else and thus before
my $q = new CGI;
The output goes to the server process' stdout and not to the HTTP stream, so the default is text/plain as you can see in the CGI output.
After you solve that problem you'll find that the output still looks like a big ugly block because you need to format and send a valid HTML page, not just a big block of text. You cannot just dump a bunch of text to the browser and expect it to do anything intelligent with it. You must create a complete HTML page with tags to layout your content, probably with CSS as well.
In other words, the output required will be completely different from the output when writing only to the terminal. How to structure it is up to you, and explaining how to do that is out of scope for StackOverflow.
As the other answers state, the BEGIN block is executed at the very start of your program.
BEGIN {
$|=1;
print "Content-type: text/html\n\n";
use CGI::Carp('fatalsToBrowser');
}
There, you output an HTTP header Content-type: text/html\n\n. The browser sees that first, and treats all your output as HTML. But you only have text. Whitespace in an HTML page is collapsed into single spaces, so all your \n line breaks disappear.
Later, you print another header, the browser cannot see that as a header any more, because you already had one and finished it off with two newlines \n\n. It's now too late to switch back to text/plain.
It is perfectly fine to have a CGI program return text/plain and just have text without markup be displayed in a browser when all you want is text, and no colors or links or tables. For certain use cases this makes a lot of sense, even if it doesn't have the hyper in Hypertext any more. But you're not really doing that.
Your BEGIN block serves a purpose, but you are overdoing it. You're trying to make sure that when an error occurs, it gets nicely printed in the browser, so you don't need to deal with the server log while developing.
The CGI::Carp module and it's functionality fatalsToBrowser bring their own mechanism for that. You don't have to do it yourself.
You can safely remove the BEGIN block and just put your use CGI::CARP at the top of the script with all the other use statements. They all get run first anyway, because use gets run at compile time, while the rest of your code gets run at run time.
If you want, you can keep the $|++, which turns off the buffering for your STDOUT handle. It gets flushed immediately and every time you print something, that output goes directly to the browser instead of collecting until it's enough or there is a newline. If your process runs for a long time, this makes it easier for the user to see that stuff is happening, which is also useful in production.
The top of your program should look like this now.
#!/usr/bin/perl -T
# middle.pl - a simple concordance
use strict;
use warnigns;
use diagnostics;
use CGI;
use CGI::Carp('fatalsToBrowser');
$|=1;
my $q = CGI->new;
Finally, a a few quick words on the other parts I deleted from there.
Your comment requires over the use statements is misleading. Those are use, not require. As I said above, use gets run at compile time. require on the other hand gets run at run time and can be done conditionally. Misleading comments will make it harder for others (or you) to maintain your code later on.
I removed the -w flag from the shebang (#!/usr/bin/perl) and put the use warnings pragma in. That's a more modern way to turn on warnings, because sometimes the shebang can be ignored.
The use diagnostics pragma gives you extra long explanations when things go wrong. That's useful, but also extra slow. You can use it during development, but please remove it for production.
The comment sanity check should be moved down under the CGI instantiation.
Please use the invocation form of new to instantiate CGI, and any other classes. The -> syntax will take care of inheritance properly, while the old new CGI cannot do that.
I ran your cgi. The BEGIN block is run regardless and you print a content-type header here - you have explicitly asked for HTML here. Then later you attemp to print another header for PLAIN. This is why you can see the header text (that hasn't taken effect) at the beginning of the text in the browser window.
I am checking ( or want to check ) if the counter gets to 5 I want to reset the counter back to one and start again until all the records have been read,I have the following code on a test page:
<?php do {
if ($count == 5) { $count = 1;}
echo $count;
echo "<div id=\"col$count\"><div id=\"col$count-content\"><img class=\"resetImg\" name=\"colpic$count\" src=\"assets/images/student_artwork/thumb_" . $row_rsILU['ilu_artwork'] . "\" alt=\"\" style=\"background-color: #999966\"><br><br><span class=\"artistName\">" . $row_rsILU['ilu_fname'] . " " . $row_rsILU['ilu_lname'] . "</span> <br><hr>
Concept Artist<br>
email#artistdomain.ca<br>
416-833-1111<br>
www.portfolio.ca<br>
</div></div>";
$count++;
} while ($row_rsILU = mysql_fetch_assoc($rsILU)); ?>
The record set has three records in it but the count always seems to stop at two and only shows the two images. I am checking the count and if it gets to 5, I want to reset it to 1 and start over again until the total records in the record set have been read. Maybe I am just getting tired but I cannot get this. Would appreciate anyones help on this.
Cheers,
Dave
According to your code, you are trying to echo the row before actually fetching it (I mean the first iteration). Why not just use while loop instead?
<?php
while ($row_rsILU = mysql_fetch_assoc($rsILU)) {
if ($count == 5) { $count = 1;}
echo $count;
echo "<div id=\"col$count\"><div id=\"col$count-content\"><img class=\"resetImg\" name=\"colpic$count\" src=\"assets/images/student_artwork/thumb_" . $row_rsILU['ilu_artwork'] . "\" alt=\"\" style=\"background-color: #999966\"><br><br><span class=\"artistName\">" . $row_rsILU['ilu_fname'] . " " . $row_rsILU['ilu_lname'] . "</span> <br><hr>
Concept Artist<br>
email#artistdomain.ca<br>
416-833-1111<br>
www.portfolio.ca<br>
</div></div>";
$count++;
}
?>
Please let me know if that solves your problem.
Humm ... well now, this is embarrassing! Turns out there was a second, older SELECT statement in the code that I missed and so it was not returning one of the three records because it was not looking for it, based on the older SELECT statement. Sorry about that but thanks for your help.
I'm running a perl script to pull a list of about 20 text files, and parse through them. For some reason my process is bombing partway through the list, and am having trouble debugging it.
Anyone know the location of the Strawberry perl log file, and if there's a builtin max execution time, or memory limit variable like in PHP?
There are three files:
1. cron.php
2. nightly_script.php
3. myscript.pl
It successfully executes the first insert statement in that while loop, but not anymore after that. Since this is running like a cron job I don't have any output window to look at. This is why I was hoping there's a log somewhere, so if there's a syntax error, or a mysql error I can see it somewhere. Also, if I just run myscript.pl on the file in question directly, it works no problem.
cron.php
date_default_timezone_set('America/New_York');
/*
min hr dom month dow cmd
hour in 24 hour format, no leading zeros
*/
$jobsQueue = Array();
$jobsQueue[] = Array('10', '0', '*', '*', '*', 'php c:\nightly_script.php'); // These items are order dependent, so run as one script that synchronously executes each command
while(1) {
$now = time();
$min = date('i',$now);
$hr = date('G',$now);
echo "$hr:$min\n";
foreach($jobsQueue AS $job) {
if($job[0] == $min && $job[1] == $hr) {
system("$job[5]>NULL");
}
}
sleep(60);
}
?>
nightly_script.php
// Process Hand Histories
system('perl myscript.pl');
?>
myscript snippet
while ( ($key, $value) = each(%players) ) {
print "$key => $value\n";
if($value > 0)
{
$uname = $key;
$uname =~ s/player(.*)(\s*)/$1/;
$connect = DBI->connect("DBI:mysql:database=$config_mysql_db;host=$config_mysql_server",$config_mysql_user,$config_mysql_pass,{'RaiseError' => 1});
print "\n*****\n$uname\n*****\n";
$updateStatement = "INSERT statement";
$executeStatement = $connect->prepare($updateStatement);
$executeStatement->execute();
$updateStatement = "UPDATE command";
$executeStatement = $connect->prepare($updateStatement);
$executeStatement->execute();
delete $players{$key};
# Clean up the record set and the database connection
$connect->disconnect();
}
elsif($value <= 0)
{
delete $players{$key};
}
}
Since perl doesn't have a log like php, you can create your own log file by redirecting perl's stdout and stderr to a file. Try doing this by modifying the system call in nightly_script.php.
system('perl myscript.pl 1>myperllog.txt 2>&1');
or
system('perl myscript.pl 1>myperllog.txt 2>myperllog.err');
My program (which happens to be in Perl, though I don't think this question is Perl-specific) outputs status messages at one point in the program of the form Progress: x/yy where x and yy are a number, like: Progress: 4/38.
I'd like to "overwrite" the previous output when a new status message is printed so I don't fill the screen with status messages. So far, I've tried this:
my $progressString = "Progress\t$counter / " . $total . "\n";
print $progressString;
#do lots of processing, update $counter
my $i = 0;
while ($i < length($progressString)) {
print "\b";
++$i;
}
The backspace character won't print if I include a newline in $progressString. If I leave out the newline, however, the output buffer is never flushed and nothing prints.
What's a good solution for this?
Use autoflush with STDOUT:
local $| = 1; # Or use IO::Handle; STDOUT->autoflush;
print 'Progress: ';
my $progressString;
while ...
{
# remove prev progress
print "\b" x length($progressString) if defined $progressString;
# do lots of processing, update $counter
$progressString = "$counter / $total"; # No more newline
print $progressString; # Will print, because auto-flush is on
# end of processing
}
print "\n"; # Don't forget the trailing newline
Say
$| = 1
somewhere early in your program to turn autoflushing on for the output buffer.
Also consider using "\r" to move the cursor back to the beginning of the line, rather than trying to explicitly count how many spaces you need to move back.
Like you said, don't print out a newline while your progress counter is running or else you will print out your progress on a separate line instead of overwriting the old line.
I know it's not quite what you asked for, but possibly better. I happened on this same problem and so rather than deal with it too much went to using Term::ProgressBar which looks nice too.
You can also use the ANSI escape codes to directly control the cursor. Or you can use Term::ReadKey to do the same thing.
I had to tackle something similar to this today.
If you don't mind reprinting the entire line, you could do something like this:
print "\n";
while (...) {
print "\rProgress: $counter / $total";
# do processing work here
$counter++;
}
print "\n";
The "\r" character is a carriage return-- it brings the cursor back to the beginning of the line. That way, anything you print out overwrites the previous progress notification's text.
I'd like to run a subcommand from Perl (or pipe it into a Perl script) and have the script process the command's output immediately, rather than waiting for a timeout, a newline, or a certain number of blocks. For example, let's say I want to surround each chunk of input with square brackets. When I run the script like this:
$ ( echo -n foo ; sleep 5 ; echo -n bar ; sleep 5; echo baz) | my_script.pl
I'd like the output to be this, with each line appearing five seconds after the previous one:
[foo]
[bar]
[baz]
How do I do that?
This works, but is really ugly:
#! /usr/bin/perl -w
use strict;
use Fcntl;
my $flags = '';
fcntl(STDIN, F_GETFL, $flags);
$flags |= O_NONBLOCK;
fcntl(STDIN, F_SETFL, $flags);
my $rin = '';
vec($rin,fileno(STDIN),1) = 1;
my $rout;
while (1) {
select($rout=$rin, undef, undef, undef);
last if eof();
my $buffer = '';
while (my $c = getc()) {
$buffer .= $c;
}
print "[$buffer]\n";
}
Is there a more elegant way to do it?
From perlfaq5: How can I read a single character from a file? From the keyboard?. You probably also want to read How can I tell whether there's a character waiting on a filehandle?. Poll the filehandle. If there is a character there, read it and reset a timer. If there is not character there, try again. If you've retried and passed a certain time, process the input.
After you read the characters, it's up to you to decide what to do with them. With all the flexibility of reading single characters comes the extra work of handling them.
Term::ReadKey can do this for you. In particular setting the ReadKey() mode to do the polling for you.
use Term::ReadKey;
$| = 1;
while( my $key = ReadKey(10) ) {
print $key;
}
If there's time inbetween each character, you might be able to detect the pauses.
Perl also does line input - if you don't use getc you should be able to add newlines to the end of foo, bar, etc and perl will give you each line.
If you can't add newlines, and you can't depend on a pause, then what exactly do you expect the system to do to tell perl that it's started a new command? As far as perl is concerned, there's a stdin pipe, it's eating data from it, and there's nothing in the stdin pipe to tell you when you are executing a new command.
You might consider the following instead:
$ echo "( echo -n foo ; sleep 5 ; echo -n bar ; sleep 5; echo baz)" | my_script.pl
or
$ my_script.pl$ "echo -n foo ; sleep 5 ; echo -n bar ; sleep 5; echo baz"
And modify your perl program to parse the input "command line" and execute each task, eating the stdout as needed.
-Adam
See How to change Open2 input buffering. (Basically, you have to make the other program think it's talking to a tty.)
You didn't mention how you are reading input in your Perl script, but you might want to look at the getc function:
$|++; # set autoflush on output
while ($c = getc(STDIN)) {
print $c;
}