Posting Gzipped data with curl - perl

Question

im trying to use the system curl to post gzipped data to a server but i keep ending up with strange errors
`curl -sS -X POST -H "Content-Type: application/gzip" --data-binary $data $url`
gives
curl: no URL specified!
and
`curl -sS -X POST -H "Content-Type: application/gzip" --data-binary "$data" $url`
gives
sh: -c: line 0: unexpected EOF while looking for matching `"'
sh: -c: line 1: syntax error: unexpected end of file

Answer 1

Adding the " is a step in the right direction, but you didn't consider that $data might contains ", $, etc. You could use String::ShellQuote to address the issue.
use String::ShellQuote qw( shell_quote );
my $cmd = shell_quote(
curl => (
'-sS',
'-X' => 'POST',
'-H' => 'Content-Type: application/gzip',
'--data-binary' => $data,
$url,
),
);
my $output = `$cmd`;
Or you could avoid the shell entirely.
my #cmd = (
curl => (
'-sS',
'-X' => 'POST',
'-H' => 'Content-Type: application/gzip',
'--data-binary' => $data,
$url,
),
);
open(my $pipe, '-|', #cmd) or die $!;
my $output = do { local $/; <$pipe> };
close($pipe);
Or if you didn't actually need to capture the output, the following also avoids the shell entirely:
system(
curl => (
'-sS',
'-X' => 'POST',
'-H' => 'Content-Type: application/gzip',
'--data-binary' => $data,
$url,
),
);
That said, I don't see how you can possibly send strings containing NUL bytes, something a gzipped file is likely to have. I think your approach is inherently flawed.
Do you know that libcurl (the guts of curl) can be accessed via Net::Curl::Easy?

Answer 2

I did not succeed in getting curl to read the data straight from stdin, but process substitution did work, for example:
curl -sS -X POST -H "Content-Type: application/gzip" --data-binary #<(echo "Uncompressed data" | gzip) $url
This technique removes any need to having to write to a temporary file first.

Answer 3

This is because your binary data contains all kind of trash, including quotes and null bytes, which confuse the shell. Try putting your data into some file and post that file.