I am using CloverETL Designer for ETL operations and I want to load some csv files from GCS to my Clover graph. I used FlatFileReader and tried to get file using remote File URL but it is not working. Can someone please detail the entire process here??
The path for file in GCS is
https://storage.cloud.google.com/PATH/Write_to_a_file.csv
And I need to get this csv file into the FlatFileReader in CloverETL Designer
You should use the Google Cloud Storage API to GET the file; Clover's HTTPConnector component will allow you to pass in the appropriate parameters to make a GET request (you will presumably have to do an OAuth2 authentication first to get a token), and send the output to a local destination specified in "Output File URL." Then you can use a FlatFileReader to read from that local file.
GCS has several different ways to download files from your buckets. You can use the console and the Cloud Storage browser. Steps: open the storage browser, navigate to the object you want to download, right click, and save to your chosen local folder. If you use Chrome the save appears as “Save Link As…”.
To use the GS Utility, use this command:
`gsutil cp gs://[BucketName]/[ObjectName] [ObjectDestination]`.
Or you can use client libraries or the REST APIs to download files. With these last options you could work with a number of files or create a job to download them. Once they are in a location known to Clover ETL the process is straightforward.
Within Clover designer, under the navigation pane you can right click a folder and choose import. Pick the one in which you placed your GCS file. Once the file is imported then you can use data from it like any other datafile in Clover. Since this is a .csv file, remember to edit your metadata (right click the component, choose extract metadata then edit inside the Metadata Editor -- for data types, labels and such.) Assign metadata to the edges of your components so they know what is coming in/going out of that step. Depending on your file, this process may be repeated many times.
Even with an ETL tool, getting the data and data types correct can be tricky. If you have questions about how to configure data types or your edges in an ETL project, a wiki may help. The web has additional resources may help you get the end analysis you’re looking for.
I'm confused about 2 questions here :
Can I embed a tableau worksheet hosted on Tableau Public in tableau dashboard(using tableau desktop) using web container component of the dashboard ?
When i save a tableau workbook with live connection connected to say oracle db and save the file as twbx file, does the twbx file still contain the entire data source(schema used in this case) in the twbx file ?
You should be able to since it will be a url. however, if your goal is to just share the report you publish, then just share the url with the users.
the twbx file will only have the metadata for the connection, essentially a connection string, and the structure of the views/tables you are querying.. it won't have a full blueprint of the database, nor should it.
I wanted to download all workbook from project without changing it's format using Tabcmd utility. Format can be TWB or TWBX.
is there any option i can get either FORMAT or WORKBOOK URL or BOTH of all tableau workbook by giving Project name, so that i can download it as backup.
I do not wanted to use POSTGRES or Tableau REST APIs. I am in need of batch script only.
There's no way of downloading all workbooks from a project using tabcmd without knowing all of the workbook names.
Although you've said you don't want use it, the REST API is the best way of doing this.
Please provide some information about difference between twb workbook with extract and twbx workbook. Also I am facing some issues, I have workbook(twbx) on Tableau Server which use published extract. Extract was refreshed today. But workbook shows old data....
TWB - XML file for your Tableau Workbook, contains all the selections and layout you've made. It does not contain data. These tend to be very small.
TWBX - zipped file that contains the TWB as well as data used by TWB in an extract
Here's some more info from the Tableau website.
http://kb.tableausoftware.com/articles/knowledgebase/sending-packaged-workbook
Try closing & opening your workbook. If that doesn't refr
Make sure that the data at the path or database connection that the Tableau Server points to the exact source you wish to refresh from.
Remember the Server may have different drives mounted, different firewall rules. If you are reading from a file like Excel or Access to create your extract; changing the version of the file elsewhere on the file system won't affect the extract on the Tableau Server if that extract points elsewhere (kind of obvious, but often forgotten, especially if a copy of the Excel file is bundled up into the twbx file).
It is also often a good idea in production to publish a data source and extract separately from the workbooks that use it so that they can be updated independently. Look under the data menu to find the publish command.
TWBX is intended for sharing. It does not link to the original file source; instead it contains a copy of the data that was obtained when the file was created.
If you need to give them TWBX, you can create a TWB as a template and then use it to create TWBX when your data source is updated. Your clients will get a TWBX that they want and you don't have to do anything.
You can even have a batch process for that. Here is a video: https://www.youtube.com/watch?v=Odk2xr6qOoQ
As Ryan mentioned, the twbx file contains its own data extract. Since you have a twbx file that uses a published data extract as its source, you basically created an extract of the original extract. In other words, the data is not coming from the published extract anymore, but is self-contained in the workbook itself, so refreshing the published extract won't update your workbook.
You can try scheduling the workbook itself (after the refresh of the extract of course). However, that didn't work for me, and I always have to refresh the extract manually from the Tableau Desktop.
I am new to using tableau server. I wanted to know if it was possible to download a tableau workbook (which is linked to a live SQL DB) with the data from the tableau server? Basically I would want to have a downloaded packaged tableau file which I can pass around for demos, without having to connect to the database every time.
You can do this programmatically in python simply.
TSC(Tableau Server Client) is the python library for tableau server.
You can publish, update, delete and download workbooks freely using TSC
Please visit https://github.com/tableau/server-client-python/blob/master/docs/docs/api-ref.md
On this page, just search by "workbooks.download", then you will get the instructions how to download a workbook with data extraction.
In essence, no. Since the workbook is using a live connection, the data is still on the SQL database and not on Tableau Server.
If you want to be able to create a packaged workbook for demo use, then you'll have to create an extract of the data that can be saved with the workbook.
There is another approach to this problem. Now that the viz is using the tableau online extract.
1.Download the workbook from the server.
2.Once done, open the dashboard in tableau desktop. You will see a data extract with a check mark on it.
3.Right click on the data extract and click on create a local copy.
4.This will extract all the data so far and stores it locally.
5.Next step is to replace the datasource from the Online extract to the local copy you have created. 6.Save the workbook as .twbx.
You are done. Hope this will work for you..
I believe what you are trying to do can be accomplished using a tableau extract. This will create a package of the sql db you want on the tableau server. the user could then make a packaged workbook or make a local copy of the extract.
http://kb.tableau.com/articles/knowledgebase/working-with-tableau-data-extracts
Yes, you can from the top tight hand corner of the view depending on your user permission levels.
When you connect tableau server data source click on extract then publish your workbook to server. when any one click on download the extracted data will also download along with worksheet
you must have the permission to download.at least Interactor
click "download", then save the workbook as "twbx" file to include data in the package file.
if your permission is "viewer", you cannot download tableau reports.
one more thing you could do is.
open workbook on server>>>>click download>>>>>select data>>>>go to full data>>> select show all column>>> download all row as text file
now download the workbook create a new data source with a text file you just downloaded. select create an extract
after completion, select replace data source
and it's done.
Unfortunately you cannot unless you have the credentials for the live db connection, if you do you can download the workbook and create a local copy of the data source by entering the credentials and then extract the data which will store the data locally. This will improve the performance as well as then you will be using Tableau extract which is optimized for reporting
With a live connection you will not be able to download the data with a Tableau workbook.
For this you can follow this pocedure :
Create an extract connection in your workbook and publish it in the server with an refresh schedule to it (As per you Data updates).
Now whenever you download the workbook it will come with the last refreshed data with it.
Hope it helps :)
make the data connection as extract, then it will copy with data copy in dashboard.
Download your workbook.
Edit your data Sources
Right-click Data Source and select "Create Local Copy"
Select the original data source .. right click it and replace it with the local copy
Lastly, right-click the original data source and close it.
There you have your workbook with a local extract(tde)
You can refresh this local extract anytime you want by refreshing the data source.
You can do so by using tableau extract file(.twbx)
But if the workbook has large data and can't create extract locally, then we can use something called empty extract.
Here's how
https://www.tableau.com/about/blog/2013/9/easy-empty-local-extracts-25152
Make an empty extract using above method and load it on server.
Let server build the full extract and then download it.
You are good to go with that workbook.
Since the workbook is using a live connection, the data is still on the SQL database and not on Tableau Server.
If you want to be able to create a packaged workbook for demo use, then you'll have to create an extract of the data that can be saved with the workbook.
Create an extract connection in your workbook and publish it in the server with an refresh schedule to it (As per you Data updates). Now whenever you download the workbook it will come with the last refreshed data with it.
You can save it as a .twbx or download from the published version