Use Case:
Given a Template.twb that uses Template.hyper I want to create Copy1.hyper and Copy2.hyper that are used in Dashboard1.twb and Dashboard2.twb using automation and the Tableau API.
I've broken this down into three steps, but I can only do the first two currently.
Write the hyper files using the Hyper API and publish to Server
using
Create and publish Dashboard1.twb and Dashboard2.twb that still use Template.hyper as a datasource.
Change datasource of Dashboard1.twb to Copy1.hyper.
I think this is done using workbooks.update_connection, but I haven't gotten it to work or found an example online.
I'm a python novice and mostly using the python scripts made available in the Hyper API and Tablea Server Client to create and publish respectively.
Is this possible?
I can change the datasource by hand in tableau desktop, but we want this automated.
I was thinking I could just append all the data and filter on some unique string generated by each query, but this seems like it would get unwieldy very fast.
Related
I'm trying to work out how I can export individual Cognos Reports via the command line, for the purposes of source versioning in Git at a report-by-report level. I presume XML would be the output format.
I read that the Cognos SDK can help but you need to build your own solution, which may be possible but this use case feels like something many others would already want and there'd be tooling already.
Of course, importing the individual report would also be needed.
Can anyone help here please?
Thanks.
If your end game is version control (Who changed what, when?), you should look into MotioCI. Last time I looked, there was no free version of MotioCI.
You can use tools like the ones provided by companies like http://www.motio.com. With the free version you can export the XML of the reports but only one by one.
You can also use a Cognos deployment of the reports that generates a zip file with the XML of the reports, but all the reports are in the same file and you will have to extract the XML of the individual reports by hand.
I found the SDK to be cumbersome and, when I got it working, slow.
Yes, report specs are XML.
I have created a process that produces output like what you are asking for. Here's what it involves:
A recursive common table expression (CTE) query to get the report
specs along with the folder structure as seen in Cognos.
A PowerShell script to run the query and write the results to the file system.
Another PowerShell script to pull the current content from the remote git repo, run the first PowerShell script, then add, commit, and push the results up to the remote git repo.
I also wrote a PowerShell script to perform the operations associated with git push. This involves using a program I found called HTML Tidy (http://tidy.sourceforge.net/) that can be used to make the XML human-readable. This helps with diffs in git. I use TFS, so I get a nice, side-by-side diff if I have tidied the XML. (Otherwise, it tells me the only line of XML has changed.)
I recently added output for dashboards (exploration) and data sets (dataSet2). Dashboards are stored as JSON, so my routine had to tidy that (simple in PowerShell).
I run my routine daily, getting new and modified content from the last 3 days (just in case), and weekly to do an entire dump (to capture the deletes). The weekly process takes about six minutes. The daily process is negligible.
Before you ask: I hesitate to provide actual code because I can't take any responsibility for your system.
Updates:
Hacking away at the Content Store database is not recommended and it is not supported by IBM.
For reference/comparison: I'm running IBM Cognos 11.0.7 on IIS on Windows 2012 R2 with the Content Store database on MS SQL Server 2016. Your system may be different.
Additional Resources
https://www.cognoise.com/index.php/topic,28289.msg113869.html#msg113869
https://www.cognoise.com/index.php/topic,17411.msg50409.html#msg50409
https://learn.microsoft.com/en-us/powershell/scripting/overview?view=powershell-6
https://learn.microsoft.com/en-us/sql/t-sql/language-reference?view=sql-server-2017
https://git-scm.com/docs
http://tidy.sourceforge.net/
I would like to change the properties of multiple diagrams together rather than clicking on them one by one. Does anyone know how this can be achieved?
You can use the scripting facility of Enterprise Architect to loop the diagrams you would like to change and update them.
See this section of the manual to get help.
There is a bunch of example scripts included with EA, either from the local scripts, or from the EAScriptLib MDG.
Another source of examples is my Github repository: https://github.com/GeertBellekens/Enterprise-Architect-VBScript-Library
You could write a SQL to manipulate your database. t_diagram.PDATA holds a long cryptic string where one part is ScalePI=0; (which is the default for no scaling). You can alter that to be ScalePI=1; (meaning scale to one page).
String manipulations vary from database to database. So you need to write your own which you can execute in a script using
Repository.Execute("UPDATE t_diagram ...")
Note that you should test this in a sandbox first since invalid SQLs can easily disrupt your whole repository.
Tableau is an excellent tool for visualizing data. However, it is designed to be the final stop in a data (ETL) pipeline.
My Tableau workbook uses a bunch of Table Calcs to generate a list of "recommended orders". Rather than view these, I want to automate and execute them. This would make Tableau the engine of a quasi-ML process.
In other words, I would like to make Tableau a part of my ETL pipeline and send data to another tier. How can I write a back-end program that executes my Tableau workbook and receives a results dataset?
See the end of this article for example data I want to automate:
http://robm26.blogspot.com/2015/10/keep-your-factory-humming-with-tableau.html
Any ideas?
You're not not going to like the answer I'm going to give you -- "Don't do this".
Tableau isn't meant to be a task in a larger ETL pipeline and the reason you're having problems making it behave the way you want is it's not meant to be done.
Above and beyond the fact that you've figured out how to get a result that you want in Tableau ("the work is done"), Tableau isn't offering you any real value in the scenario you're describing. Use a tool (like Alteryx) that is really purpose built for this sort of work.
The above answer is correct that tabcmd is the way to pull it out. We use a function in python to generate the tabcmd requests so that they can be batched.
import subprocess
def runTabCmd(cmd):
# run tableau command and display the output
print cmd
if run_tabcmd == 'yes':
p = subprocess.Popen(
cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in p.stdout.readlines():
print line
You probably already knew that, but for us it was a way to completely automate the pulling and loading into another python package like scikit-learn for a streamlined ML solution
I'm editing this answer to agree with Russell's answer. Tableau is not an ETL tool and should not be used as such. If you absolutely have to do something, you can use what I provided. Otherwise, the best practice is to use a tool designed for the job.
You can easily use tabcmd to get the results of a view in CSV, which can be used later in your ETL process. If you need to automate it, you can write a script and execute it with a cron job. I, myself, have a few views that are exported to CSV and used later in my ETL stream to feed our CRM.
Just remember to create the view exactly as you want it to be exported to CSV - usually including the order of the fields. Another tip is that I don't let it use the default "Measure Names" and "Measure Values" - to make sure everything is good on my CSV, I have the fields added manually in the row/columns section.
I am trying to create custom build version number.
We already using some custom build template and have created custom activities. Now I am trying to add custom build version activity in our template.
My idea is to select last build and extract only number which represent number of builds. This is last digit and it can be e.g. 5.2.3.1032 (marked as bold).
Based on last digit I will create next build number and insert into TFS database.
I can also use UpdateBuildNumber activity to insert number, but is there way to select build number by querying TFS db directly? It is also good if is possible via PowerShell.
Thanks
It sounded familiar, in fact I wrote a post on this topic a while ago. Probably the code and details must be reviewed to work with recent TFS, but the general idea is still applicable.
At the core you use the IBuildServer.QueryBuilds method to read your build history and pick the information you need.
I am working on a Ruby Script (using MacRuby with Scripting Bridge) to do some processing on a FileMaker Pro database (FMP Advanced 10.) I am able to read databases, tables, and records by creating a FileMakerProAdvancedApplication object:
framework 'scriptingbridge'
fm = SBApplication.applicationWithBundleIdentifier('com.filemaker.client.advanced')
the resulting object works great for reading values out of FileMaker databases, but I am confused about how to create new objects. The FileMaker scripting dictionary provides a "create" command, but it does not show up in the header generated by sdef /Applications/FileMaker\ Pro\ 10\ Advanced/Filero\ Advanced.app/ | sdp -fh --basename FilemakerProAdvanced (command taken from Apple's Scripting Bridge Docs.) Is it possible to create new elements with FMP's script support? What am I missing?
Not sure I know to much about the scripting bridge, but assume that it must be using AppleScript behind the scenes. When you say create new objects, to you mean records or tables?
I'm fairly certain you can't create tables (or fields) in FileMaker via AppleScript.
You can create (and delete) records within existing tables. I would fire up a copy of the AppleScript Editor, and have a look at the FileMaker script dictionary from that end.
The generated header files rarely duplicate the Dictionary as seen via Applescript for an application. There are sometimes duplicate function calls and/or some objects and functions that are available via Applescript are not available vis Scripting Bridge. As far as I know, there is nothing to indicate why this change would be in place and there is no way to get around this limitation.