I have a simple test setup:
A SQL Server (2017) with one database, with one table
A SQL Server Analysis Server (2017, with compatibility level 1400)
I have created a simple tabular model in Visual Studio with one datasource (the database with one table) and one table
This is my power query:
let
Source = #"SQL/MYCOMPUTER\SQLDEV;SampleDatabase",
dbo_testTable = Source{[Schema="dbo",Item="testTable"]}[Data]
in
dbo_testTable
I have deployed this tabular model to my SSAS instance...
Now my question: if the table in my SQL Server is updated (added records), how can I see these updates reflected in the Tabular Model? Do I have to rerun the Tabular Model somehow?
I have tried "Process Table" in SSMS on the Tabular model table, but it does not get the new records...
Processing a table processes whichever dimension or fact table you selected and this will only read data from the database objects used by this table. What processing is actually performed will depend on the type of processing that you used. As far as the question in the answer you posted, Process Full on an entire Tabular model will remove all data from the deployed model, then reload everything and process the hierarchies and measures as well, so yes the new data from the underlying tables will now be in the model for all tables within it after you processed it using this option. There are multiple processing types that can either be done at the database, table, or partition level. You can view additional details on these via the Microsoft reference.
I have found that on the level of the Database in the SSAS instance, there is an option "Process Database" that has an option "Process Full", which does update all the underlying tables.
But maybe there is a better way to do this?
Related
I created an entity-relationship diagram (logical model) and engineered it to a relational model.
The tables were generated. Now I need to use them from the connection XE as you see in the picture.
The tables I made can only be seen on the data modeler design view in the "Browser", how do I get them on the connection "XE" to generate data dictionary, etc?
There are three possibilities:
you just need to expand the tables item in the tree to see your tables
You are looking in the wrong schema/user - go down to Other Users, and find the schema those tables belong to
The tables do not exist in the current database
If #3 is the issue, you would need to create them, possibly using the information in your Data Model - that is, you can generate the DDL/SQL scripts for those tables.
Then taking those scripts, run them while connected to the appropriate database/schema.
Disclaimer: I'm an Oracle employee and the product manager for these tools.
We are working on a audit system where auditor are given access to transaction processed in last quarter. Auditor performs various analysis on the data to find out invalid/erroneous transactions that have some exceptions.
Generally, these analysis requires data to be present on some charts to view the out-layers or sometime duplication detection are done based on multiple columns.
Sometime exception detection algorithm are pretty involved that require multiple processing steps using stored procedure.
Please note that analysis rarely involves aggregation on huge rows.
Occasionally , they can change some data if they find it missing or incorrect.
We are evaluating row based (sql & nosql databases) and column store (like data warehouse systems).
Is this a use case for datawarehouse or row based store, like nosql or some RDBMS?
In short, requirements are:
- Occasional update
- Mostly read queries over last 3/months of data
- Reading data my require several messaging steps, like creating temp table in step 1, forming join with another table in step rule, delete some rows ect.
Thanks
For your task, it does not really matter how the data is stored. You need to think instead how to create a solid dimensional model, populate it with data properly, and what reporting tools to use.
To give you an example, here are a couple of common setups I've used in my projects:
Microsoft stack setup:
SQL Server for data storage
SSIS for data ETL (or write your own stored procedures if you know what you are doing)
Publish dimensional model on the same SQL Server. If your data set is large (over billion records), use SSAS Tabular instead
Power Pivot or Power BI for interactive reporting, or SSRS for paginated reports.
Open-source setup:
PostgreSQL for data storage
Use stored procedures and/or Python to process data
Publish dimensional model to another PostgreSQL database. If your data is large, publish the dimensional model to Redshift or
other columnar database
Use Tableau or Power BI for interactive reporting, or build your own reporting interface.
I think NoSQL database is a wrong choice here because audit will require highly structured data.
I have around 25 tables to load to target with same structure and which use the same logic for loading. I have prepared one job which does that, but it's a long process to design all the tables.
Is there any way to pass the table name and load to target, basically a small job (in size).
I am using Talend open studio.
Check my answer to a similar question where I proposed a generic solution for loading a MySQL table to another MySQL table.
You just need to modify the queries that retrieve the tables' metadata (columns) depending on your database type.
I was creating MySQL database to add medicine.I created a table and I need to add one more tabe.After creating it I tried to query the database from the sql workbench.But it donot show the table but it is present in the EER Model.How can I solve this problem.
Modeling is just the task of abstractly designing your schema and its objects (e.g. tables, views etc.). It does not actually create these objects. For this you have to forward engineer your model to a server (see Database menu). Once done you can use the Synchronization feature to update either model or server (or both) with any changes made.
But keep in mind this is only for the objects, not for any data.
Originally, I used Data Modelling in MySQL Workbench to design a database consisting of a series of tables (i.e. the columns and relationships).
Then using Database -> Forward Engineer, I created a database, and inserted data into the tables.
Now I've realised that the model I've designed needs some changes, and so I've altered some tables by inserted columns. My question is, how do I get MySQL Workbench to alter the tables?
Using Database -> Synchronize Model, Update Source just generates a bunch of CREATE TABLE IF NOT EXISTS sql statements, and as the tables exist, nothing changes.
What you are looking for is in the model menu Database / Synchronize model.
As I couldn't get get File -> Export -> Forward Engineer SQL ALTER Script to work, so I made a backup of the data, dropped the tables, recreated them, and then imported the data. I'd rather find a way to get MySQL Workbench to generate ALTER commands from the changes in my model
The 2011 answer is no longer up to date. I struggled to find the option in a recent version. Here is the new procedure (works for MySQLWorkbench 6.2 at least):
When you have finished editing your model, open Database -> Synchronize with Any Source
In the step Select Source you have 3 parts
Source : choose Model Schemadata
Destination : choose Live Database Server
Send updates to : choose whether the live database should be updated or if you only want to saves the changes to a .sql file
Proceed in the wizard, you can then review the tables and sql queries that will be executed. You can also ignore the update of some tables.