Upload TAR contained file to REST PUT API using PHP CURL without extracting - rest

Question

Basically, I am facing the following issue:
I have a TAR container (no compression) of a large size (4GB). This TAR encloses several files:
file 1
file 2
file 3 - the one THAT I NEED (also very large 3 GB)
other files (it doesn't matter how many.).
I should mention that I do know where the file 3 starts (start index) and how large it is(file length) because the TAR format is relatively easy to parse.
What I need to do is upload file 3 by using PHP Curl to a REST API.THE API endpoint is HTTP PUT and the headers are correctly set (it works if I'm uploading the entire TAR file).
So, INFILE = TAR Container.
File 3 starts at the Xth Byte and has a length of Y bytes. I already know the X and Y value.
I need the curl to start sending data from X to Y.
What I did until now was:
$fileHandle = fopen($filePath, "rb"); //File path is the one of the TAR archive
fseek($fileHandle, $fileStartIndex, SEEK_CUR);
And the settings of the curl are.
curl_setopt($curlHandle, CURLOPT_PUT, 1);
curl_setopt($curlHandle, CURLOPT_BINARYTRANSFER, 1);
curl_setopt($curlHandle, CURLOPT_INFILE, $fileHandle);
curl_setopt($curlHandle, CURLOPT_INFILESIZE, $fileSize);
I must mention that extracting file 3 to disk is not an option at this moment as the disk space is the main purpose of the task.
My first idea was to look at the CURLOPT_READFUNCTION, but the callback of this option should return a string (in my case very large one :3 GB, and it breaks the PHP variable size limit).
Has anyone succeeded in handling this kind of upload? Any other tips and trick about CURLOPT_READFUNCTION are also best appreciated.
Thank you!

Answer 1

According to the PHP curl doc:
CURLOPT_READFUNCTION
A callback accepting three parameters. The
first is the cURL resource, the second is a stream resource provided
to cURL through the option CURLOPT_INFILE, and the third is the
maximum amount of data to be read. The callback must return a string
with a length equal or smaller than the amount of data requested,
typically by reading it from the passed stream resource. It should
return an empty string to signal EOF.
So a combination of CURLOPT_INFILE to give curl the file handle, CURLOPT_INFILESIZE to tell curl how big the final file will be and CURLOPT_READFUNCTION to allow curl to read from the file looks like it should do what you need.
Although curl will call your CURLOPT_READFUNCTION with a $length parameter, you're free to return what you want, within the rules:
The callback must return a string with a length equal or smaller than
the amount of data requested
so if you return less than $length, curl will keep calling your CURLOPT_READFUNCTION until it returns EOF (an empty string). So you need to keep track of where you are in your file when reading in CURLOPT_READFUNCTION and start from the last read position on each call.