I'm using gwtupload.client.MultiUploader to upload zip files on the server in GWT. Then on the server I transform zip file to array of bytes to insert into database. As the result 50% of files in database are corrupted. Here a little bit of my code.
#UiField(provided = true)
MultiUploader muplDef;
public MyClass(){
muplDef = new MultiUploader();
muplDef.setValidExtensions("zip");
muplDef.addOnFinishUploadHandler(onFinishUploaderHandler);
muplDef.addOnCancelUploadHandler(onCancelUploaderHander);
}
private final IUploader.OnFinishUploaderHandler onFinishUploaderHandler = new IUploader.OnFinishUploaderHandler() {
#SuppressWarnings("incomplete-switch")
#Override
public void onFinish(IUploader uploader) {
switch (uploader.getStatus()) {
case SUCCESS:
attachZip = true;
}
}
};
private final IUploader.OnCancelUploaderHandler onCancelUploaderHander = new IUploader.OnCancelUploaderHandler() {
#Override
public void onCancel(IUploader uploader) {
attachZip = false;
}
};
Byte Array
String fileName = "D:\1.zip";
File f = new File(fileName);
byte[] edocBinary = new byte[(int) f.length()];
RandomAccessFile ff;
try {
ff = new RandomAccessFile(f, "r");
ff.readFully(edocBinary);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
My questions are: Files can be correpted even if I have OnFinishUploaderHandler and case:SUCCESS? There are other cases like ERROR, maybe this case will check the file? Or the problem is with the transformation to byte array? Can you provide me some advices, thanks.
As you said, you had two steps :
1- Uploading the zip file
2- Inserting the zip file in the database
If the step 1 is done correctly, you're gonna get a success showing that the file is transfered correcly from client to the server, what you do after that is not managed by GwtUpload.
So I guess the corruption happened when you try to insert the file in the database. If you are using MySQL Try this http://www.codejava.net/java-se/jdbc/insert-file-data-into-mysql-database-using-jdbc
Related
I have a file which is in Parquet format. I want to read it and save it in HDFS or AWS S3 in Protobuf format using spark with Scala. I am not sure of any way. Searched many blogs but could not understand anything, can anyone help?
You can use ProtoParquetReader, which is ParquetReader with ProtoReadSupport.
Something like:
try (ParquetReader reader = ProtoParquetReader.builder(path).build()
) {
while ((model = reader.read()) != null){
System.out.println("check model " + "-- " + model);
...
}
} catch (IOException e) {
e.printStackTrace();
}
In order to read from parquet you need to use the following code :
public List<Record> read(Path path) {
List<Record> records = new ArrayList<>();
ParquetReader<Record> reader = AvroParquetReader<Record>builder(path).withConf(new Configuration()).build();
for (Record value = reader.read(); value != null; value = reader.read()) {
records.add(value);
}
return records;
}
Writing to a file from parquet would be something like this. Although this is not the protobuf file this might help you get started. Have in mind that you will have issues if you end up using spark-stream with protobuf v2.6 and greater
public void write(List<Record> records, String location) throws IOException {
Path filePath = new Path(location);
try (ParquetWriter<Record> writer = AvroParquetWriter.<GenericData.Record>builder(filePath)
.withSchema(getSchema()) //
.withConf(getConf()) //
.withCompressionCodec(CompressionCodecName.SNAPPY) //
.withWriteMode(Mode.CREATE) //
.build()) {
for (Record record : records) {
writer.write(record);
}
} catch (Exception e) {
e.printStackTrace();
}
}
public static void write(byte[] aInput, String aOutputFileName, String dirName) {
(new File(dirName)).mkdir();
try {
OutputStream output = null;
try {
output = new BufferedOutputStream(new FileOutputStream(dirName + "/" + aOutputFileName));
output.write(aInput);
} finally {
output.close();
}
} catch (FileNotFoundException ex) {
System.out.println("File not found.");
} catch (IOException ex) {
System.out.println(ex);
}
}
The code above is from a library i'm using and it is supposed to create an output file and write a byte array to it. I checked logcat and saw the Strict Mode Policy violation Write.toDisk. I understand what it is supposed to be for my questions are: (1) Does Strict mode prevent you from doing disk reads and write on the main thread? (2) Does that mean that the file or folder were not actually created? (3) How then would I go about creating a folder or file within my App that doesn't trigger this? (4) What is the recommended way to handle disk read / write off the main ui thread, a real world example would be appreciated
Thanks in Advance
(1) It turns out that Strict mode doesn't actually prevent you from making writes to the disk it just gives a warning. From Android Developer "StrictMode is a developer tool which detects things you might be doing by accident and brings them to your attention so you can fix them". https://developer.android.com/reference/android/os/StrictMode
(2) The files were actually being created it's just that I was just not familiar with writing and reading from disk
(3) There are numerous ways to go about creating files (i) first you get a hold of a file directory to write the file to:
context.getFilesDir()
(ii) then you get an outputstream writer (iii) then you write out the data with the writer
public void makeFile(String filename){
//Create temp file for filename
FileOutputStream fos = null;
try {
fos = new FileOutputStream(new File(filename));
fos.write(filename.getBytes());//Write the contents of the file to app folder
fos.flush();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}finally {
if(fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
(iv) finally you close the outputstream writer
(4) The recommended way is to use either an AsyncTask or some other background running class like FutureTask or to use Threads or Runnable:
public class DownloadFileThread implements Runnable{
public void run(){
//your code here
}
}
I am able to download a single file but how I can download a zip file which contain multiple files.
Below is the code to download a single file but I have multiples files to download. Any help would greatly appreciated as I am stuck on this for last 2 days.
#GET
#Path("/download/{fname}/{ext}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response downloadFile(#PathParam("fname") String fileName,#PathParam("ext") String fileExt){
File file = new File("C:/temp/"+fileName+"."+fileExt);
ResponseBuilder rb = Response.ok(file);
rb.header("Content-Disposition", "attachment; filename=" + file.getName());
Response response = rb.build();
return response;
}
Here is my working code I have used response.getOuptStream()
#RestController
public class DownloadFileController {
#Autowired
DownloadService service;
#GetMapping("/downloadZip")
public void downloadFile(HttpServletResponse response) {
response.setContentType("application/octet-stream");
response.setHeader("Content-Disposition", "attachment;filename=download.zip");
response.setStatus(HttpServletResponse.SC_OK);
List<String> fileNames = service.getFileName();
System.out.println("############# file size ###########" + fileNames.size());
try (ZipOutputStream zippedOut = new ZipOutputStream(response.getOutputStream())) {
for (String file : fileNames) {
FileSystemResource resource = new FileSystemResource(file);
ZipEntry e = new ZipEntry(resource.getFilename());
// Configure the zip entry, the properties of the file
e.setSize(resource.contentLength());
e.setTime(System.currentTimeMillis());
// etc.
zippedOut.putNextEntry(e);
// And the content of the resource:
StreamUtils.copy(resource.getInputStream(), zippedOut);
zippedOut.closeEntry();
}
zippedOut.finish();
} catch (Exception e) {
// Exception handling goes here
}
}
}
Service Class:-
public class DownloadServiceImpl implements DownloadService {
#Autowired
DownloadServiceDao repo;
#Override
public List<String> getFileName() {
String[] fileName = { "C:\\neon\\FileTest\\File1.xlsx", "C:\\neon\\FileTest\\File2.xlsx", "C:\\neon\\FileTest\\File3.xlsx" };
List<String> fileList = new ArrayList<>(Arrays.asList(fileName));
return fileList;
}
}
Use these Spring MVC provided abstractions to avoid loading of whole file in memory.
org.springframework.core.io.Resource & org.springframework.core.io.InputStreamSource
This way, your underlying implementation can change without changing controller interface & also your downloads would be streamed byte by byte.
See accepted answer here which is basically using org.springframework.core.io.FileSystemResource to create a Resource and there is a logic to create zip file on the fly too.
That above answer has return type as void, while you should directly return a Resource or ResponseEntity<Resource> .
As demonstrated in this answer, loop around your actual files and put in zip stream. Have a look at produces and content-type headers.
Combine these two answers to get what you are trying to achieve.
public void downloadSupportBundle(HttpServletResponse response){
File file = new File("supportbundle.tar.gz");
Path path = Paths.get(file.getAbsolutePath());
logger.debug("__path {} - absolute Path{}", path.getFileName(),
path.getRoot().toAbsolutePath());
response.setContentType("application/octet-stream");
response.setHeader("Content-Disposition", "attachment;filename=supportbundle.tar.gz");
response.setStatus(HttpServletResponse.SC_OK);
System.out.println("############# file name ###########" + file.getName());
try (ZipOutputStream zippedOut = new ZipOutputStream(response.getOutputStream())) {
FileSystemResource resource = new FileSystemResource(file);
ZipEntry e = new ZipEntry(resource.getFilename());
e.setSize(resource.contentLength());
e.setTime(System.currentTimeMillis());
zippedOut.putNextEntry(e);
StreamUtils.copy(resource.getInputStream(), zippedOut);
zippedOut.closeEntry();
zippedOut.finish();
} catch (Exception e) {
}
}
I have a CAS consumer AE which is expected to iterates over CAS objects in a pipeline, serialize them and add the serialized CASs to an xml file.
public class DataWriter extends JCasConsumer_ImplBase {
private File outputDirectory;
public static final String PARAM_OUTPUT_DIRECTORY = "outputDir";
#ConfigurationParameter(name=PARAM_OUTPUT_DIRECTORY, defaultValue=".")
private String outputDir;
CasToInlineXml cas2xml;
public void initialize(UimaContext context) throws ResourceInitializationException {
super.initialize(context);
ConfigurationParameterInitializer.initialize(this, context);
outputDirectory = new File(outputDir);
if (!outputDirectory.exists()) {
outputDirectory.mkdirs();
}
}
#Override
public void process(JCas jCas) throws AnalysisEngineProcessException {
String file = fileCollectionReader.fileName;
File outFile = new File(outputDirectory, file + ".xmi");
FileOutputStream out = null;
try {
out = new FileOutputStream(outFile);
String xmlAnnotations = cas2xml.generateXML(jCas.getCas());
out.write(xmlAnnotations.getBytes("UTF-8"));
/* XmiCasSerializer ser = new XmiCasSerializer(jCas.getCas().getTypeSystem());
XMLSerializer xmlSer = new XMLSerializer(out, false);
ser.serialize(jCas.getCas(), xmlSer.getContentHandler());*/
if (out != null) {
out.close();
}
}
catch (IOException e) {
throw new AnalysisEngineProcessException(e);
}
catch (CASException e) {
throw new AnalysisEngineProcessException(e);
}
}
I am using it inside a pipeline after all my annotators, but it couldn't read CAS objects (I am getting NullPointerException at jCas.getCas()). It looks like I don't seem to understand the proper usage of CAS consumer. I appreciate any suggestions.
i am using GWTUpload to upload a file.
i am getting the server info, file name, content type etc.. in onFinishHandler, but there's no option to get the file content from server to client.
Note : am trying to upload XML File and EXCEL File
i am using GWT 2.4, GXT 3.0.1, GWTUpload 0.6.6
here's the sample code
Client Code - OnFinishHandler
u.addOnFinishUploadHandler(new OnFinishUploaderHandler() {
#Override
public void onFinish(IUploader uploader) {
if (uploader.getStatus() == Status.SUCCESS) {
System.err.println(uploader.getServerResponse());
UploadedInfo info = uploader.getServerInfo();
System.out.println("File name " + info.name);
System.out.println("File content-type " + info.ctype);
System.out.println("File size " + info.size);
System.out.println("Server message " + info.message);
}
}
});
Servlet Code
public class GWTFileUploadServlet extends UploadAction {
private static final long serialVersionUID = -6742854283091447922L;
String fileContent;
File uploadedFile;
#Override
public String executeAction(HttpServletRequest request,
List<FileItem> sessionFiles) throws UploadActionException {
String response = "";
int cont = 0;
for (FileItem item : sessionFiles) {
if (false == item.isFormField()) {
cont++;
try {
File file = File.createTempFile("upload-", ".bin");
item.write(file);
uploadedFile = file;
fileContent = item.getContentType();
response += "File saved as " + file.getAbsolutePath();
} catch (Exception e) {
throw new UploadActionException(e.getMessage());
}
}
}
removeSessionFileItems(request);
return response;
}
#Override
public void getUploadedFile(HttpServletRequest request,
HttpServletResponse response) throws IOException {
if (fileContent != null && !fileContent.isEmpty()) {
response.setContentType(fileContent);
FileInputStream is = new FileInputStream(uploadedFile);
copyFromInputStreamToOutputStream(is, response.getOutputStream());
} else {
renderXmlResponse(request, response, XML_ERROR_ITEM_NOT_FOUND);
}
}
}
please help me....
You can read the file you have created in the filesystem when you called item.write(...), but it is better if you get an InputStream from the FileItem you received and write its content anywhere. For instance if the content is a String you can use a StringWritter to get it:
InputStream inputStream = item.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(inputStream, writer);
String theContentString = writer.toString();
[EDITED]
To get the content of the file in client side, so you have to retrieve it from the server using any method:
As as a customized message in your gwtupload servlet if the content of the file is ascii: use return String of executeAction.
Do a RequestBuilder call to the servlet to get the file using the uploader url value.
Use any GWT ajax mechanism like RPC.
In modern browsers you can get the content of a file in client side without sending it to server side. Take a look to lib-gwt-file
In your code You can just use
byte[] file ;
file = item.get();
And You will get all the file content in an encoded format in the "file" variable .