I want to download an Excel file from Sharepoint and use it as input for a tFileInputExcel component. tHttpRequest seems to be the only component able to download files from a sharepoint server with authentication.
What of course works is to download the file via tHttpRequest, write the response to a file, and load that file with the tFileInputExcel component.
Example, but I don't want it like that.
However, I do not want an intermediate file and use the "stream" functionality of the tFileInputExcel component. What I do not achieve is to convert the downloaded content into a stream that is usable by the excel component:
((java.io.InputStream)globalMap.get("what goes here? And where does it come from?"))
Here's an example using the tFileFetch component, but this component is not able to download from sharepoint with authentication.
You can do it by writing the content returned by tHttpRequest to a ByteArrayOutputStream inside a tJavaFlex, then converting it to a ByteArrayInputStream like this :
Make sure you uncheck "Write response content to file" on tHttpRequest.
In my example I download a text file because I don't have a link to an excel, but it works the same. You can read your file as an input stream like this :
I extended the tHttpRequest component and created tHttpRequestEx. The new component saves a ByteArrayInputStream to the global map which can be read from the tFileInputExcel component:
((java.io.ByteArrayInputStream)globalMap.get("tHttpRequestEx_1_INPUT_STREAM"))
I uploaded the component on exchange.talend.com.
Of course this has the big disadvantage to use a non-standard component. It may be the better solution to stick with the file download, and use tCreateTemporaryFile to deal with the complexity of creating and deleting a file.
Related
I am using CloverETL Designer for ETL operations and I want to load some csv files from GCS to my Clover graph. I used FlatFileReader and tried to get file using remote File URL but it is not working. Can someone please detail the entire process here??
The path for file in GCS is
https://storage.cloud.google.com/PATH/Write_to_a_file.csv
And I need to get this csv file into the FlatFileReader in CloverETL Designer
You should use the Google Cloud Storage API to GET the file; Clover's HTTPConnector component will allow you to pass in the appropriate parameters to make a GET request (you will presumably have to do an OAuth2 authentication first to get a token), and send the output to a local destination specified in "Output File URL." Then you can use a FlatFileReader to read from that local file.
GCS has several different ways to download files from your buckets. You can use the console and the Cloud Storage browser. Steps: open the storage browser, navigate to the object you want to download, right click, and save to your chosen local folder. If you use Chrome the save appears as “Save Link As…”.
To use the GS Utility, use this command:
`gsutil cp gs://[BucketName]/[ObjectName] [ObjectDestination]`.
Or you can use client libraries or the REST APIs to download files. With these last options you could work with a number of files or create a job to download them. Once they are in a location known to Clover ETL the process is straightforward.
Within Clover designer, under the navigation pane you can right click a folder and choose import. Pick the one in which you placed your GCS file. Once the file is imported then you can use data from it like any other datafile in Clover. Since this is a .csv file, remember to edit your metadata (right click the component, choose extract metadata then edit inside the Metadata Editor -- for data types, labels and such.) Assign metadata to the edges of your components so they know what is coming in/going out of that step. Depending on your file, this process may be repeated many times.
Even with an ETL tool, getting the data and data types correct can be tricky. If you have questions about how to configure data types or your edges in an ETL project, a wiki may help. The web has additional resources may help you get the end analysis you’re looking for.
I want to compress a file before uploading it on server in sapui5 (image(png/jpeg/jpg)/pdf/). I want to upload a large files but i want to compress that file before uploading. Please suggest me solution ?
I think you will not be able to do it with instant upload. Instead, you should obtain the file from the change event of the file uploader (check out the change event, it has a files parameter).
You can then use that file object together with the zip.js library to create an in-memory zip and save it into e.g. a Blob. Afterwards you simply send the blob into a POST request (e.g. look at How can javascript upload a blob?). Maybe you should also provide some file-uploading specific headers (like the Slug).
I have this problem:
I want to make an interface where the user can drag and drop file uploads. Filepicker.io provides an easy solution to this in the form of filepicker.makeDropPane. However, this feature does not support parallel uploads without the callbacks getting confused.
So I would like to write my own drag/drop interface using standard html5 listeners, and make a direct api upload call to filepicker.io with the actual file data in string form. This way I can write the management of parallel uploads on my own.
Does filepicker.io have an api call that would allow me to do this? I only see these two things:
1) File selection modal
2) Auto drag-drop features
I don't see a way to simply upload a file directly from file data.
What can I do?
You should make use of the filepicker.store() command, which accepts a DOM file object. For example:
var input = document.getElementById("store-input");
filepicker.store(input, function(FPFile){/*your code here*/});
More details available at https://developers.filepicker.io/docs/web/#store
i'm beginner with GXT and i'm wondering if there is a way to parse a file and extract some informations without uploading it.
i created a formpanel that contains an uploadFile form but i don't know waht's next, how to get the complete path of the file so i can read/write with java io or how to retrieve the file or is there an alternatif solution, thank you.
Best Regards.
You can do it in some modern browsers using bleeding edge HTML5 apis for which you would need to use GWT JSNI code. There are no api's from GWT team as is.
HTML5 FileReader
FileReader includes four options for reading a file, asynchronously:
FileReader.readAsBinaryString(Blob|File) - The result property will contain the file/blob's data as a binary string.
FileReader.readAsText(Blob|File, opt_encoding) - The result property will contain the file/blob's data as a text string.
FileReader.readAsDataURL(Blob|File) - The result property will contain the file/blob's data encoded as a data URL.
FileReader.readAsArrayBuffer(Blob|File) - The result property will contain the file/blob's data as an ArrayBuffer object.
Example of GWT wrapper over these -
https://github.com/bradrydzewski/gwt-filesystem
You can read about it more from here - How to retrieve file from GWT FileUpload component?
IMHO you cannot read it .
Due to security reasons javascript(gwt) doesn't have access to the system drives files.
http://en.wikipedia.org/wiki/JavaScript#Security
see Opening a file in local file system in javascript
In order to get the file you need to make a server call.
Instead you can do your validation server side and throw proper messages to user.
P.S : i am not considering modern browser concept.What happens if someone opened in other than so called modern browsers?? Will the programm runs same?? Its always better to do server side validation..
Greetings,
I'm looking for a way to launch a GWT module when a user clicks on an XML file and have the module consume the xml data. Ideally I would like to render the XML in a rich manner and would prefer to use GWT controls instead of having to lay it out by hand via xslt + javascript.
I'm supposing one way would be to point the xml to a well known xslt that creates a simple html page that forces a redirect to the gwt module but how would I transfer the xml data to said module to allow for enhanced formatting?
Another way would be to have the process that produces the xml also include the bootstrap gwt module but it would be creating multiple bootstrap instances over time and pollute the user's directory.
The use case is that a user would run this app on their local machine which outputs an XML file. If they try and view the xml file in a browser, I'd like to have the GWT module take over and present the data accordingly. I would rather they not have to go to a page and upload the data manually.
Appreciate any ideas on the matter.
TIA
If it's something that runs on the user's machine, I would recommend to ship an executable, or generate a parallel HTML file to present the data. JavaScript run from file:/// will not be able to acces the filesystem.