In our current setup, we have a separate project for each customer. The customers all have the same dashboards but different data sources. Whenever we change a dashboard of one customer, we have to replicate the changes to all the other customers. Currently, this is a manual process, where we re-upload the changed workbook for each customer and replace the connected data sources.
I now want to automate that process by using tableaus rest API in combination with their document api. For the data sources, we are not using hyper files instead of having a live connection to a database.
This is my current implementation:
tableau_auth = TSC.TableauAuth('admin', 'tableau-admin')
server = TSC.Server('http://localhost:8080')
with server.auth.sign_in(tableau_auth):
# download original workbook
server.workbooks.download(workbook_id=source_work_book_id, filepath="tmp", include_extract=False)
source_wb = Workbook('tmp/source.twbx')
# get datasources which have connections
wb_datasources = [s for s in source_wb.datasources if len(s.connections) != 0]
# mappings to map datasource from source to target datasources
# (all the datasources have the same name but are in different projects)
dbname_to_datasource = {d.content_url: d.name for d in datasources if d.project_id == source_project_id}
source_to_target = {d.name: d.content_url for d in datasources if d.project_id == target_project_id}
# overwrite source datasources
for datasource in wb_datasources:
datasource.connections[0].dbname = source_to_target[dbname_to_datasource[datasource.connections[0].dbname]]
source_wb.save_as("tmp/target.twbx")
# write back to tableau server
new_workbook = TSC.WorkbookItem(target_project_id)
server.workbooks.publish(new_workbook, file_path="tmp/target.twb", mode=TSC.Server.PublishMode.CreateNew)
I get the following error when uploading it:
400011: Bad Request
There was a problem publishing the file 'target.twbx'.
It doesn't even work if I re-upload the workbook unmodified. If I change the download to include the extract file I can upload the unmodified workbook successfully:
server.workbooks.download(workbook_id=source_work_book_id, filepath=tmp_dir, include_extract=True)
The only problem with that is that I'm now adding unnecessary data and it also doesn't solve my issue of how to replace the data sources.
Does anyone have an idea what I'm doing wrong or is there an alternative way to do this?
Related
Hi would just like to ask if this is possible, I am currently working on ADF, what I want to do is get workitems from analytics.dev.azure.com/[Organization]/[Project] then copy it to SQL Database. i am currently already doing this for 1 project, but want to do it for multiple projects without creating multiple copyto tasks within ADF but just run a Lookup to ForEach to iterate through all the team analytics URLs, is there anyway to do this?
We can use lookup and for-each activity to copy data to SQL dB tables from all URLs. Below are the steps
Create a lookup table which contains the entire list of URLs
Next in for each activity's settings, type the following in items for getting output of lookup activity
#activity('Lookup1').output.value
Inside for each activity, use copy activity.
In source, create a dataset and http linked service. Enter the base URL and relative URL. I have stored relative URLs in lookup activity. Thus I have given #{item().url} in relative URL
In sink, Create azure SQL database table for each item in for each activity or use the existing tables and copy data to those tables.
We are deploying Tableau for a bank.
We had created 6 test dashboards using dummy data on a staging data base using sql connection and lets say has an ip 10.10.10.10.
Now we need to use the same view we had used with the dummy data on Live data but using a different connection which is again an sql engine & IP lets say as 20.20.20.20. All the variable names and other properties are the same, not difference is that the Live data would not have calculated fields which we can deploy on the Live environment.
The challenge is: the LIVE data being of a bank is highly confidential and cannot be used from outside operations site rather we need to deploy it from an ODC [restricted environment]. Hence we simply cannot do a replace data source.
Hence we are planning to move twbx files and data extracts for each of these views using a shared folder to the ODC. Then the process would be like below:
As the LIVE sql data base is different from the dummy sql we will get error
We will select edit data connection
Will select tableau data extract for each sheet and dashboard
Will then select the option of replace data source and select LIVE SQL database
Will extract the new data
The visualization should work fine
Earlier we had just moved TWBX files hence it failed. Is there a different approach to it.
I did something similar to it
For that, you must have
same schema as of Live database and dummy database
do not change name of any source table or column
create your viz
send it in the .tbw form which is editable HTML format
Now the hard part- open your tbw in notepad and replace all connection details to new one
save and open in the tableau
tell me if it didn't worked
One method would be to modify your hosts file on your local computer, pointing the production server name the staging instance of the database. For example, let's say your production database is prod.url.com and you have a reporting staging db server instance called reportstage.otherurl.com
Open your hosts file. Add an entry for prod.url.com. Point it to reportstage.otherurl.com
Develop the report in Desktop, with the db connection string to prod.url.com.
When you publish the twb file to Server, no connection string changes are needed.
Another easier way is to publish the twb to Server with your staging connection string but edit the connection string in the data source in Server.
Develop the twb file on your local computer against your staging database.
Publish the twb file to Server.
Go to the workbook on Server and instead of looking at the views, click on Data Sources.
Edit the data source(s) connection information. This allows you to edit the server name, port, username, or password.
I've used this second method quite a bit. We have an environment where we can't hit the production db outside of the data center. Our staging environment doesn't have that restriction. We develop against the stage db, deploy, and edit the server name in the data source.
We have three Organization tenents, Dev, Test and Live. All hosted on premise (CRM 2011. [5.0.9690.4376] [DB 5.0.9690.4376]).
Because the way dialogs uses GUIDs to refference record in Lookup, we aim to maintain GUIDs for static records same across all three tenents.
While all other entities are working fine, I am failing to import USERS and also maintain their GUIDS. I am using Export/Import to get the data from Master tenent (Dev) in to the Test and Live tenents. It is very similar to what 'configuration migration tool' does in CRM 2013.
Issue I am facing is that in all other entities I can see the Guid field and hence I map it during the import wizard but no such field shows up in SystemUser entity while running import wizards. For example, with Account, I will export a Account, amend CSV file and import it in the target tenant. When I do this, I map AccountId (from target) to the Account of source and as a result this account's AccountId will be same both in source and target.
At this point, I am about to give up trying but that will cause all dialogs that uses User lookup will fail.
Thank you for your help,
Try following steps. I would strongly recommend to try this on a old out of use tenant before trying it on live system. I am not sure if this is supported by MS but it works for me. (Another thing, you will have to manually assign BU and Roles following import)
Create advance find. Include all required fields for the SystemUser record. Add criteria that selects list of users you would like to move across.
Export
Save file as CSV (this will show the first few hidden columns in excel)
Rename the Primary Key field (in this case User) and remove all other fields with Do Not Modify.
Import file and map this User column (with GUID) to the User from CRM
Import file and check GUIDs in both tenants.
Good luck.
My only suggestion is that you could try to write a small console application that connects to both your source and destination organisations.
Using that you can duplicate the user records from the source to the destination preserving the IDs in the process
I can't say 100% it'll work but I can't immediately think of a reason why it wouldn't. This is assuming all of the users you're copying over don't already existing in your target environments
I prefer to resolve these issues by creating custom workflow activities. For example; you could create a custom workflow activity that returns a user record by an input domain name as a string.
This means your dialogs contain only shared configuration values, e.g. mydomain\james.wood which are used to dynamically find the record you need. Your dialog is then linked to a specific record, but without having the encode the source guid.
Using .NET, I'd like to connect to an EA model from an external application.
If I have more than one EA model open -- eg two remote SQL Server hosted models -- how do I specify extracting data from only one of the models?
# any way to specify a specific data source?
var r = new EA.Repository();
# As I don't think is what I want because:
# 1) didn't want to Open a document -- just connect to it
# 2) don't have a filename - just a model name and/or ConnString...
bool isOpen = r.OpenFile("C:/Sparx-EA/Test Project.EAP");
# etc.
Element ele = r.GetElementByID(10);
Thank you!
This is documented in the manual:
OpenFile (string Filename) Boolean
Notes: This is the main point for
opening an Enterprise Architect project file from an automation
client, and working with the contained objects.
If the required
project is a DBMS repository, and you have created a shortcut .eap
file containing the database connection string, you can call this
shortcut file to access the DBMS repository.
You can also connect to a
SQL database by passing in the connection string itself instead of a
filename.
A valid connection string can be obtained from the Open
Project dialog by selecting a recently opened SQL repository.
Parameters: Filename: String - the filename of the Enterprise
Architect project to open
Repository.OpenFile() is the correct method to use. You can pass it a connection string, it doesn't have to be a file.
A Repository object can only be connected to one EA project at a time. So in order to connect to two EA projects in the same process, create two Repository objects.
Finally, the numeric identities used in calls like Repository.GetElementByID() are valid only within a repository. The number 10 likely refers to two different elements in two repositories -- or might have been deleted in one of them. If you know you've got the same element in two repositories, use Repository.GetElementByGuid() instead.
I have a requirement where I have to read data from sql server local database and first map it in XML file provided by another third party org. who have their own database. Then once I have proper mapping of fields I have to transform the data from sql server database to XML format and vice versa.
So far, I am able to connect sqlserver database in mirthconnect however I dont know what steps are required to create in channels and transformer to carry the task of reading data and mapping corresponding fields to XML format provided by third party and finally writing in XML file provided and vice versa.
In short if I can get details of creating such channel in mirth connect where I can read sql server database and map the fields in corresponding xml file....I guess I can write to it. Same way applies if I go from xml format to sqlserver database. Can someone tell me how to accomplish this?
For database field mapping whats the best way to map fields entirely on two different databases is there any tool which can help....
Also once the task of transforming the data from one end to another is accomplished is there any way of validation in mirth connect that verifies that data is correctly moved from one to another?
If you want to process one row at a time, the normal database reader will work fine; just set the data type under Summary to XML for all steps. Set a destination of channel writer to nowhere and run it once to see what it does in the Dashboard. You can copy and paste that as an example into your message template so you can map variables.
If you want to work an entire result at one time in the Transformer steps, I find it easier to create a custom reader and use "FOR XML RAW, ELEMENTS" on the end of my Microsoft SQL query.
Something like:
//build connection
var dbConn = DatabaseConnectionFactory.createDatabaseConnection('com.microsoft.sqlserver.jdbc.SQLServerDriver','jdbc:sqlserver://servername:1433;databaseName=dbname;integratedSecurity=true;','',''); //this uses the MS JDBC driver and auth dll
//query results with XML output from server 'FOR XML' statement at end
var result = dbConn.executeCachedQuery("SELECT col1 AS FirstColumn, col2 AS SecondColumn FROM [dbname].[dbo].[table1] WHERE [processed] = 'False' FOR XML RAW, ELEMENTS");
//Make sure we are at the top of results
result.beforeFirst();
//wrap XML. Namespace etc. not required
XMLresult = '<message>';
//XML broke up across several rows in one column. Re-combine
while (result.next()) {
XMLresult += result.getString(1);
}
XMLresult += '</message>';
dbConn.close();
return XMLresult;