Write files from multiple rest requests - rest

Question

I have a rest service written to receive a file and save it.
The problem is that when I receive more than 2 requests, the files are not written only the last request is taken into consideration and written.
Here is my code:
#POST
#RequestMapping(value = "/media/{mediaName}/{mediaType}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
#ResponseBody
public String updateResourceLocally(#FormDataParam("rawData") InputStream rawData, #PathVariable("mediaName") String mediaName, #PathVariable("mediaType") String mediaType) {
logger.info("Entering updateResourceLocally for " + jobId + "; for media type: " + mediaType);
final String storeDir = "/tmp/test/" + mediaName + ("/");
final String finalExtension = mediaType;
final InputStream finalRawData = rawData;
// new Thread(new Runnable() {
// public void run() {
// writeToFile(finalRawData, storeDir, finalExtension);
// }
// }).start();
writeToFile(finalRawData, storeDir, finalExtension);
// int poolSize = 100;
// ExecutorService executor = Executors.newFixedThreadPool(poolSize);
// executor.execute(new Runnable() {
// #Override
// public void run() {
// writeToFile(rawData, storeDir, finalExtension);
// }
// });
logger.info("File uploaded to : " + storeDir);
return "Success 200";
}
I tried to put the writeToFile into threads, but still no success. Here is what writeToFile does
public synchronized void writeToFile(InputStream rawData,
String uploadedFileLocation, String extension) {
StringBuilder finalFileName = null;
String currentIncrement = "";
String fileName = "raw";
try {
File file = new File(uploadedFileLocation);
if (!file.exists()) {
file.mkdirs();
}
while (true) {
finalFileName = new StringBuilder(fileName);
if (!currentIncrement.equals("")) {
finalFileName.append("_").append(currentIncrement).append(extension);
}
File f = new File(uploadedFileLocation + finalFileName);
if (f.exists()) {
if (currentIncrement.equals("")) {
currentIncrement = "1";
} else {
currentIncrement = (Integer.parseInt(currentIncrement) + 1) + "";
}
} else {
break;
}
}
int read = 0;
byte[] bytes = new byte[1024];
OutputStream out = new FileOutputStream(new File(uploadedFileLocation + finalFileName));
while ((read = rawData.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
} catch (IOException e) {
throw new RuntimeException(e.getMessage());
}
}
The writeToFile creates a folder and writes a file, if the file already exists, it appends 1 and then increments the 1 accordingly and writes the file, so I would get raw.zip, raw-1.zip, etc.
I think the inputstream bytes are being lost, am I correct in my assumption?
NOTE: I do not have a UI client, I am using Poster a Firefox extension.
Update: What I am trying to achieve here is very simple
I receive number of requests with files attached
I need to save them. If the mediaName and mediaType are the same, then I need to append something to the filename and save it in the same location
If they are different I do not have a problem
The problem I am facing with the current code is that, when I post multiple time to the same URL, I have file-names created according to what I want, but the file content is not right, they vary depending on when the request came in and only the last POST's request is written properly.
Eg. I have a zip file of size 250MB, when I post 5 time, the 1st four will have random sizes and the 5th will have the complete 250MB, but the previous four should also have the same content.

Answer 1

You must separate the stream copy from the free filename assignation. The stream copy must be done within the calling thread (jersey service). Only the file naming operation must be common to all threads/requests.
Here is your code with a little refactoring :
getNextFilename
This file naming operation must be synchronized to guarantee each call gives a free name. This functions creates an empty file to guarantee the next call to work, because the function relies on file.exists().
public synchronized File getNextFilename(String uploadedFileLocation, String extension)
throws IOException
{
// This function MUST be synchronized to guarantee unicity of files names
// Synchronized functions must be the shortest possible to avoid threads waiting each other.
// No long job such as copying streams here !
String fileName = "raw";
//Create directories (if not already existing)
File dir = new File(uploadedFileLocation);
if (!dir.exists())
dir.mkdirs();
//Search for next free filename (raw.<extension>, else raw_<increment>.<extension>)
int currentIncrement = 0;
String finalFileName = fileName + "." + extension;
File f = new File(uploadedFileLocation + finalFileName);
while (f.exists())
{
currentIncrement++;
finalFileName = fileName + "_" + currentIncrement + "." + extension;
f = new File(uploadedFileLocation + finalFileName);
}
//Creates the file with size 0 in order to physically reserve the file "raw_<n>.extension",
//so the next call to getNextFilename will find it (f.exists) and will return "raw_<n+1>.extension"
f.createNewFile();
//The file exists, let the caller fill it...
return f;
}
writeToFile
Must not be synchronized !
public void writeToFile(InputStream rawData, String uploadedFileLocation, String extension)
throws IOException
{
//(1) Gets next available filename (creates the file with 0 size)
File file = getNextFilename(uploadedFileLocation, extension);
//(2) Copies data from inputStream to file
int read = 0;
byte[] bytes = new byte[1024];
OutputStream out = new FileOutputStream(file);
while ((read = rawData.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
}