I have started working with Windows Workflow recently (the VS2010 RC / .NET 4.0 version) and am stuggling to get to grips with the SQL persistence functionality.
I have managed to attach persistence to my WorkflowServiceHost via an SqlWorkflowInstanceStoreBehavior object and in my database there are rows appearing in the [System.Activities.DurableInstancing].[InstancesTable] table.
However, I don't know how to make sense of any of this data (it seems as though quite a few columns are in binary format). How can I store custom data regarding my workflow in this? How do I retreive this from the table for MI style reporting?
I can't seem to find any info on the web regarding storing custom data (and then retrieving it again) - please help :)
Many thanks in advance!
The data you see is all use by the workflow persistence system and not really suitable for your own consumption. If you want to query on your own data you need to use a mechanism called property promotion that stores the data in a queryable format using the InstancePromotedProperties table. Basically you need to implement a PersistenceParticipant and overwrite the CollectValues() function to add the values to one of the collections.
See here for more details.
Related
I am using Telosys tools for code generation. It's very nice tool and help me a lot.
But there is one problem, that is, it provides database schema information and i can access in templates (templates are formerly velocity templates), which is good, but how can i get selected entity's data from database? There is no way i can find, by which i can get that selected table data.
Please provide solution if any, or provide alternate way to do it.
Thanking You!
Telosys Tools is designed to retrieve the model from the database,
not the data stored in the tables.
But it allows to create your own specific tooling classes usable
in the templates, so it's possible to create a specific Java Class to retrieve the data from the database.
There's an example of this kind of specific class in the "database-doc" bundle
https://github.com/telosys-tools/database-doc-bundle-TT210 ( in classes folder )
To simplify the loading the simplest way is to create the class in the "default package" (no java package)
NB:
The problem is that the jar containing the JDBC driver
is not accessible by the generator class-loader, so you will have to use a specific class-loader and to connect directly with the JDBC driver.
Here is an example : https://gist.github.com/l-gu/ed0c8726807e5e8dd83a
Don't use it as is (the connection is never closed) but it can be easily adapted.
This is a long question, but I would be very very thankful if I can get some good advice on this. In short, I’m looking for a good approach for version upgrade of MS SQL database schema that also demands data being moved from deleted tables into new tables.
I think Stack Overflow is the most appropriate place for this question (not dba.stackexchange.com) because at its core, this is an issue for .NET developers using Entity Framework, and the database parts of this consists mostly of auto-generated sql scripts.
Background
A .NET application and SQL database is running in Azure (The application in worker roles and the database in Azure SQL). Until now, version upgrades have worked fine, because all database schema changes have been simple (like adding a new column). However, from now on I also need to deal with moving data from one table to another during upgrades. (I’m able to fix this temporarily by creating a new database, generate a script with data from the old database and manually edit the script to make it fit the new schema, but I hope there is a better approach).
I use Entity Framework and I use Model First. Entities and associations are defined in Visual Studio Data Model Designer, and this approach is very appropriate for my application.
I use a dacpac to upgrade the Azure SQL database, and this approach has worked well until now (but now I will get data loss, so now I must find a way to move data to new tables).
I hope I can continue to use entity framework and defining entities/associations in the designer, but it’s fine to switch away from dacpac upgrade to another technology if needed.
Upgrade approach until now
I add new entities (tables), associations (relations) and properties (columns) in the designer.
I right-click, pick “Generate Database from Model…” and this results in a .sql script that drops old database objects and creates the new database objects.
I create an empty database and run the script to create the tables/keys etc.
In SQL Server Management Studio, I right-click the database and pick “Tasks -> Extract Data-tier Application…”. When the wizard completes I get the dacpac I need (Actually I can now delete the database, since I only created it to be able to get the dacpac file, since I don’t think I can generate it in Visual Studio Data Model Designer).
I right-click the Azure SQL database and pick “Tasks -> Upgrade Data-tier Application…” and follow the wizard.
Until now I have never had data loss, so this has worked fine!
Current situation
This is a simplified example to illustrate the issue, but I will get into almost identical situations quite often from now on it seems. Look at the old and the new version of the schema in the figure below. Assume there is already data in the database. I need the data in ImageFile to end up in ImageFileOriginal or ImageFileProcessed depending on the IsOriginal boolean/bit value. Using “Upgrade Data-tier Application” I will get alerted of data loss. What approach would you recommend to deal with this? As I said earlier, it’s fine to switch away from dacpac upgrade to another technology if needed.
I have read about Visual Studio Database Projects, Fluent Migrator, Red Gate and Entity Designer Database Generation Power Pack (It doesn't support Visual Studio 2012), but I didn’t find a good way for this. I admit I haven’t spent a whole day digging into each technology, but I certainly spent some time to try finding a good approach.
The best way to migrate database schema (create / delete tables / columns) and also data, is using the SSDT - Sql Server Data Tools, available for Visual Studio 2010 and Visual Studio 2012.
Here are some very useful links:
http://msdn.microsoft.com/data/tools
http://blogs.msdn.com/b/ssdt
http://msdn.microsoft.com/en-us/data/hh297027
In the Configuration class set the constructor as below:
public Configuration()
{
AutomaticMigrationsEnabled = true;
AutomaticMigrationDataLossAllowed = false;
}
Set the AutomaticMigrationEnabled property to true
means we are using automatic code first migration and another property AutomaticMigrationDataLossAllowed is set to false, means that during the migration no existing data is lost from that migration of the table of the database.
The entire Configuration class is as follows.
I'm developing an Lighswitch application in VS2012 with an external SQL Server 2008. I'm using a WCF RIA Service to fetch my data, using these techniques: http://www.c-sharpcorner.com/UploadFile/raj1979/how-to-use-wcf-ria-service-in-lightswitch-2012/.
My main table contains some columns with static data and about 5 columns with foreign keys. The static data columns loads instant but all the foreign properties takes 3-5 secs to load on a screen with ~100 rows. This is not acceptable..
The problem for me is that I don't know where the bottleneck might be. Any ideas where I could start looking? The only thing I could think of so far is that my ADO.NET model uses Lazy Loading, and it does.
All the foreign key properties only has two columns with a PK and some text, so they're not data heavy.
Any help appreciated, thanks!
I don't know if this is the exact answer to your speed problem, but that article you quoted suggests that you add a new ADO.NET Data Model to the class library containing the RIA Service.
There's absolutely no reason to generate a second data model, as LightSwitch already generates one when you attach the external data source (your SQL Server database). You may as well re-use it. An EF data model takes time to construct, so I suppose that using two EF data models could explain the slowness you're experiencing, but it could also be something else entirely.
Have a look at this article (the original RIA Service/LightSwitch article, & still the best), which among several other techniques, Eric will show you how to re-use the data model that LightSwitch creates for your data source.
How Do I: Display a Chart Built On Aggregated Data
In this article, the RIA service is created to specifically aggregate data to display in a chart, but you can do whatever you need to do in it. It doesn't have to aggregate data. I use RIA services with an attached data source (a SQL database) extensively, to provide data for lists, reports, as well as lookup data for combo boxes.
I am almost a beginer at this new art, but as Yan says in his book, instead a very good one, lightswitch is directed to the screen. I dare to ask you if you need to have 100 lines in your screen. You can't even see them all. If you reduce that number to 20 you will reduce your waiting time to 1/5th. I've had the same problem and now I use the least possible number of rows in screen. For that to be possible without disturbing the business performance, I choose to open screens with the maximum number of parameters, not required, which I send when I open them. If one of the parameters is not the option in the case I mention it as "Nothing". And so, I can have a minimum number of lines in the grid, fullfiling the purpous of teh screen, the program performance and the business performance.
What is the best practice for upgrading the database using ORM (DevExpress XPO, NHibernate or MS Entity Framework)?
I'm starting a new project and have to pick an ORM. The development process requires of releasing intermediate test builds quite often and likely that each build will have changes in the database structure. Each new version has to upgrade the DB gently to keep current data.
For old solutions I would provide a set of SQL scripts for upgrading the database from v1 to v2, from v2 to v3, etc. and execute them sequentially.
But how is it going to work for ORM? Should I still write SQL scripts to upgrade the DB?
I understand that simple adding new fields wouldn't cause a problem (e.g. see UpdateSchema() method for XPO), but what if I have to split a table and reallocate current records into 2 new tables?
I can't comment on the other ORM's, but I have used DevExpress XPO for a corporate treasury application since 2007. The schema changes a little with every release but there have also been some big schema changes over the years as well. A somewhat extended version of the default XPO upgrade mechanism has comfortably catered for all the changes.
There is good basic information here about upgrading XPO applications.
DevExpress provide a DBUpdater tool to assist you with the task of upgrading production environments. You can extend this tool to cater for additional requirements. In my application, we have added some options for logging, preview with rollback, etc.
Each module has virtual UpdateDatabaseBeforeSchemaUpdate() and UpdateDatabaseAfterSchemaUpdate() methods. You can significantly control the upgrade process within these.
As you mention, some of the upgrade will be handled automatically by XPO (e.g., adding a new column), but some things need additional control such as initialising the new column with a default value for existing records.
For instance, let's say MyNewField has been added to the MyEntity XPO class in version 2.0 of your application. Let's say it should default to a value of 3 for existing records. XPO will handle the creation of the new column but existing records will be NULL. (If you specify a default value in the XPO class it would only pertain to new records). In order to correct the value for existing records you would add something like the following to entity module's overridden UpdateDatabaseAfterSchemaUpdate():
public override void UpdateDatabaseAfterUpdateSchema()
{
base.UpdateDatabaseAfterUpdateSchema();
if (CurrentDBVersion < new Version(2, 0, 0, 0))
ObjectSpace.GetSession().ExecuteNonQuery(
"UPDATE [MyEntity] SET [MyNewField] = 3 WHERE [MyNewField] IS NULL");
}
(You could also use ObjectSpace.GetObjects<MyEntity>() and a foreach if you prefer to avoid the direct SQL.)
In your more extreme example of splitting a table in two, you can use the same method, but you would override UpdateDatabaseBeforeUpdateSchema() instead, run the SQL to split the table, let XPO perform any other schema updates and, if necessary, populate any default values in the UpdateDatabaseAfterUpdateSchema().
You will find that you bump into constraint problems e.g., foreign key violations so you might find you need to write some general routines such as DropAllForeignKeyConstraints() as part of the UpdateDatabaseBeforeUpdateSchema(). Sometimes you find that XPO already provide something, sometimes not. Missing constraints and indexes will get regenerated in the schema update. (In my experience switching a master data table's primary key turned out to be the hardest update routine to get right.)
By default the calls all happen in an SQL transaction so if anything fails it should all roll back.
The developers need to be aware of when a change to the domain model is likely to cause a problem with the underlying schema.
For testing, we keep a few old customer databases and run a bunch of before-and-after tests as part of the build process to make sure that existing customers are able to upgrade properly whatever version they are upgrading from. In production whenever we run into a problem upgrading, the problem data is added into this test library to prevent similar problems in the future.
We are dealing with major international companies and banks. The customers are quite happy with the result. In situations where a corporate's DBA needs to sign off on the changes, they don't seem to mind having a command line tool to do the upgrade rather than a script.
Most migration solutions can handle easy tasks, like adding new column, relationship or removing one, but fail to work when you rename a column (is that an add? or a remove following an add which equals a rename? What should you do with the data in that case?)
All three solutions have basic migrations support, XPO even lets you run your own scripts as a part of the process (to insert static/test/contant data, etc.)
There's also the MigratorDotNet project that you can use and not to rely on any ORM specific feature regarding migrations.
Personally, I would use auto migration only in dev/test environment and would have full set of upgrade scripts when running on client specific database to say upgrade from v1 to v2.
How is it going to work for ORM? Should I still write SQL scripts to
upgrade the DB?
Clear answer of this question should be on Programmer's stackexchange thread - What are the criteria for evaluating an ORM for.NET?, there i got simple answer for your question that you asked and matches with my experience with ORM while developing some project with Entity framework and Code smith ORM templates.
How does the ORM manages changes in the data model? what if I have to split a table and reallocate current records into 2 new tables?
Some can update the DB automatically within a certain measure, other
don't do anything and you'll have to do the dirty work yourself; other
provide a framework for handling change that lets you control database
updates. That means every couple of days someone needs to spend an hour updating the model to add a table or change datatypes that are changing
Ref:
https://softwareengineering.stackexchange.com/questions/6543/what-are-the-benefits-of-using-database-abstraction-by-orm
https://softwareengineering.stackexchange.com/questions/41739/best-arguments-for-against-introducing-orm-technology-into-a-companies-dev-proce/41833#41833
If you ask - what is the best practice for upgrading the db using ORM - my answer is: Don't use it if your application is more than a hobbyist app.
There are a lot of scenarios where many ORMs are unable to provide support to your specific database needs, e.g. in creating stored procedures, create indices and views or even indexed views/materialized tables without writing sql scripts. Problems like adding a new non-nullable column to an existing table are much harder to solve in ORM-Migration-Code than by writing SQL scripts.
Current Tools like Visual Studio Data Tools do handle these kind of problems way better.
As the title says, how do I view the SQL generated by Entity Framework from within my code? I'm running into an error where the EF is crashing because a field is generated by the database (a DateTime field), and I thought I set it to know that the store is generating it via StoreGeneratedPattern, but it's still crashing, so I would like to see what exactly it's trying to push up to the database.
P.S. I've only been using EF for about an hour now... Switching from L2S.
Since you don't have Sql Profiler, your best choice would be LINQPad. You can use your existing assembly.
Click Add connection -> Use a typed data context from your own assembly -> Entity framework and select your dll.
You can write queries directly against your model (or copy-paste from your code). Select the SQL 'tab' under the query window to view the generated SQL code.
You can use the Entity Framework Profiler (EFProf). It's not free, but there's a 30-day trial available. It does a lot more neat stuff besides showing you the SQL statements.
Generally, you should always use SQL Profiler to see the SQL statements that being submitted by EF into your database.
Also, I think you misunderstood about what StoreGeneratedPattern is. If you look at its possible values inside the model, you'll see that it has identity meaning that the value will be generated (by the database) when the row is inserted and will not otherwise change. The other options are Computed, which specifies that the value will be generated on inserts and updates, and None, which is the default.
So EF will not generate that DateTime field on the fly for you, you need to manually create it and then update your model from database so that EF will generate appropriate metadata to work with it at runtime.
The free AnjLab Sql Profiler will work if real SQL Profiler is not available because you're using SQL Server Express: http://anjlab.com/en/projects/opensource/sqlprofiler. It's not quite as nice as the real thing but it gets the job done well enough.
One solution would be to capture the network traffic and have a look at the data on that level. Microsoft Network Monitor does a good job of this.
Of course, that only works if you're using a separate DB server, and the connection is not encrypted.