I'm looking into a way to download a file from a url and immediately stream it (upload) it to another url (location). I don't want to dump it into disk or download the whole file in memory.
I'm new to Spring Reactive API. I came up with something like this in Kotlin:
val buff = WebClient.create("downloadUrl").get().retrieve().bodyToFlux(DataBuffer::class.java)
WebClient.create("uploadUrl").post().body(buff)
body above is org.springframework.web.reactive.function.client.body
But I'm not sure if I'm even on the right path. What is the right way of doing it?
Related
I want to download an Excel file from Sharepoint and use it as input for a tFileInputExcel component. tHttpRequest seems to be the only component able to download files from a sharepoint server with authentication.
What of course works is to download the file via tHttpRequest, write the response to a file, and load that file with the tFileInputExcel component.
Example, but I don't want it like that.
However, I do not want an intermediate file and use the "stream" functionality of the tFileInputExcel component. What I do not achieve is to convert the downloaded content into a stream that is usable by the excel component:
((java.io.InputStream)globalMap.get("what goes here? And where does it come from?"))
Here's an example using the tFileFetch component, but this component is not able to download from sharepoint with authentication.
You can do it by writing the content returned by tHttpRequest to a ByteArrayOutputStream inside a tJavaFlex, then converting it to a ByteArrayInputStream like this :
Make sure you uncheck "Write response content to file" on tHttpRequest.
In my example I download a text file because I don't have a link to an excel, but it works the same. You can read your file as an input stream like this :
I extended the tHttpRequest component and created tHttpRequestEx. The new component saves a ByteArrayInputStream to the global map which can be read from the tFileInputExcel component:
((java.io.ByteArrayInputStream)globalMap.get("tHttpRequestEx_1_INPUT_STREAM"))
I uploaded the component on exchange.talend.com.
Of course this has the big disadvantage to use a non-standard component. It may be the better solution to stick with the file download, and use tCreateTemporaryFile to deal with the complexity of creating and deleting a file.
I want to know if there is a 'right' way to make file uploads through custom tools.
I've seen the https://confluence.sakaiproject.org/display/BOOT/File+Uploads+with+RSF guide and it seens ok, but It stops with the file in memory with no further info. I can built a random file upload code but I want to make it Sakai-friendly (Using ContentHosting and Resources service?)
Any hints?
Thanks
The link you provided for the first part is a good example of how to get the upload initially processed. Going through RequestFilter will get your files validated, but you can use whatever method you want to upload it.
For the second part, I'd look at the ContentHosting webservice (createContentItem) for an example of how to add a file from a byte[] in memory after you've uploaded it.
These methods in ContentHostingService also accept InputStream as a parameter as of 2.7 (KNL-325), so you don't have to store the entire file in memory and can stream it as you're uploading, which you should do if the files are of any reasonable size.
I am new to GWT.
I want to load a large text file (50 MB) from GWT client side and output the file content in a textarea.
I tried Requestbuilder and I passed response.getText() to a string. I am able to do this for a 10-12 MB file but then it just hangs. I think it has something to do with some maximum limit of string. I can not pass the output of response.getText() to a file because then I would not be able to read that file from GWT client side as I'd need bufferreader and all.
I don't know how to make server chunk the file and send one by one responses.
Can anybody please help me with it!
Although the best option will be a server servlet to split the file so as the client could show it paginated, another option is to make the browser natively deal with the big data.
Create an iframe whose source is the url of the file in the server. If the server sends the correct headers (text/plain) the browser will show the content correctly.
Frame f = new Frame("path_to_myfile.txt");
f.setSize("600px", "400px");
RootPanel.get().add(f);
i'm beginner with GXT and i'm wondering if there is a way to parse a file and extract some informations without uploading it.
i created a formpanel that contains an uploadFile form but i don't know waht's next, how to get the complete path of the file so i can read/write with java io or how to retrieve the file or is there an alternatif solution, thank you.
Best Regards.
You can do it in some modern browsers using bleeding edge HTML5 apis for which you would need to use GWT JSNI code. There are no api's from GWT team as is.
HTML5 FileReader
FileReader includes four options for reading a file, asynchronously:
FileReader.readAsBinaryString(Blob|File) - The result property will contain the file/blob's data as a binary string.
FileReader.readAsText(Blob|File, opt_encoding) - The result property will contain the file/blob's data as a text string.
FileReader.readAsDataURL(Blob|File) - The result property will contain the file/blob's data encoded as a data URL.
FileReader.readAsArrayBuffer(Blob|File) - The result property will contain the file/blob's data as an ArrayBuffer object.
Example of GWT wrapper over these -
https://github.com/bradrydzewski/gwt-filesystem
You can read about it more from here - How to retrieve file from GWT FileUpload component?
IMHO you cannot read it .
Due to security reasons javascript(gwt) doesn't have access to the system drives files.
http://en.wikipedia.org/wiki/JavaScript#Security
see Opening a file in local file system in javascript
In order to get the file you need to make a server call.
Instead you can do your validation server side and throw proper messages to user.
P.S : i am not considering modern browser concept.What happens if someone opened in other than so called modern browsers?? Will the programm runs same?? Its always better to do server side validation..
I know it's possible to load a webpage inside eclipse, but this isn't what I'm looking for. What I'm looking for is to load raw data (json or xml) from a remote url via http. For example the plugin could request data for bugs with status unresolved. That query will go to the url which will return data. My plugin needs to then read this data. So any suggestions what I need to get this done?
new URL(url).openStream() will open an input stream. you an read data from this stream.
You may want to have a look at REST(Representational state transfer).