Is there any way to migrate and synchronize data between an on-premise SQL database and SQL Azure database apart from sync framework?
The sync framework works fine for small databases, but when it comes to large databases its not working. Is there any possible way to migrate and synchronize using change datacapture and SSIS?
you might want to clarify what you mean by "its not working". Are you having issues during the initial sync or incremental sync? what's the sync direction required for your sync? upload/download/bidirectional? do you get an error?
there are many ways to do the migration,
for synchronization, apart from Sync Framework, you may also look at Sql Azure Data Sync (largely based on Sync Framework too).
and yes, you can use Change Data Capture and SSIS if you want but note that Sql Azure doesnt have the same Change Data Capture feature as Sql Server so you'll be fine using CDC and SSIS for on-premise to Sql Azure sync only. You have to figure out another way to do change tracking on the Sql Azure side.
Related
Is it possible to migrate from SQL DB small plan to premium plan? Assume we have started with the small plan and data exceeds 10GB. Can the plan be migrated to premium? If yes, does this include data movement?
No, currently there's no way to automatically migrate your data. You have to manually migrate from the small to the premium plan. You should be able to use the SQL Database console to manually export your data, then once you've subscribed to the premium plan, you should be able to load the data via the SQL DB Console as well.
The SQL Database console does not have ability to import/export data. To move data from one SQLDB instance to another, consider to use the Bluemix DataWorks Data Load REST API.
https://www.ng.bluemix.net/docs/services/dataworks1/index-gentopic1.html#task_d4j_q1r_np
Alternatively, you may also create a Bluemix app so you may import and export data from/to SQL Database service:
http://www.ibm.com/developerworks/cloud/library/cl-sqldb-app/
I have an on-premise database. At the same time I have the database on cloud. When the on-premise database gets updated the SQL Azure database should also get updated. Only the changed fields should be updated. The rest should remain the same. How can this be achieved in minimal time?
There is a No code solution called Data SYNC CTP2 but you need to request access which unfortunately stopped for now. (http://connect.microsoft.com/sqlazurectps)
You could try using SYNC framework. Have a look at this article: http://blogs.msdn.com/b/sync/archive/2010/08/31/sql-server-to-sql-azure-synchronization-using-sync-framework-2-1.aspx
just a note. Neither Sync Framework or Sql Azure Data Sync does column level change tracking or synchronization. When a column in a row is changed, the entire row is sent during synchronization.
As Paras mentioned, Sql Azure Data Sync is in CTP stage (CTP2 now, with CTP3 supposed to come out this summer).
Sync Framework 2.1 however already supports synching with Azure.
check out Synchronizing with SQL Azure using Sync Framework
for links to various walkthroughs/samples
I have an app the runs at the store level extracting data out of a POS system. This app asks the POS system for data which in return produces some .DBF files. The app loads the DBFs in memory and saves it to SQL Server 2008 Express at the store level. This happens at 3 different stores.
I have been looking into the Microsoft Sync Framework, but have not come across any good examples of how to sync tables in one direction only, from each of the stores into a single database at the corporate level.
The data at each one of the stores is being managed by the app(delete, update, insert).
Does anyone know a good article, that I could read about synchronizing SQL data?
Thank you.
Try the following link
http://www.codeproject.com/KB/database/DTS_SQLExpress.aspx
I am creating a simple application and getting stuck with data storage option. To begin with I would like to use SQL Server as my data storage. I will not be using any special features of SQL Server, its pure tables with CRUD operations.
Now I should be able to switch the underlying data store to either SQL Data Services or Amazon S3 by changing few configuration parameters.
Is this possible??? If yes, can anyone provide high level guidance on how to go about it? Do I need to use Entity Framework to begin with SQL Server? Does Entity Framework supports SQL Data Services? Any common component which supports both SQL Data Services and Amazon S3?
Too many questions!!!
Thanks for the help in advance.
The closest ORM I know of is LightSpeed. I've never used it though. Personally, if the end goal is to use cloud storage, I'd probably just use cloud storage from the get go...
If you are going to use Amazon's SimpleDB, M/Gateway has a open source db that mimics their API.
I'm looking for advices and suggestions on how to synchronise data between two databases.
The first database is a SQL Server 2008 Express that run on disconnected laptops (no network or internet access). The second database (main) is a VFP 9.0 that run on a server.
When the user connect their laptop on the network, I want the synchronisation process to go through.
Other than the different database engines, I have the following items to take into account:
The tables don't necessary have the same structure
The primary keys are not the same (GUID in the SQL Server and often a combination of character fields in VFP)
Synchronisation of the tables must be done in a certain order to respect the parent-child relationships
On some insert on the SQL Server side, a new primary key must be generated and synchronised in the VFP table
A bunch of validations must be made and some feedback from the user are sometimes needed
Not all records need to be synchronised
Some records on the SQL Server need to be deleted after the syncronisation
Need to take into account deleted records from both side
Minimal modifications need to be done on the VFP database
There are probably other points I'm forgotting now, but I think you get the idea of the challenge I face. My guess right now are that I will need to build a custom synchronisation module, but I want your input before I go on in case I overlooked some options and to get some tips on how to approach this.
I looked rapidly at Microsoft Sync Framework, but with all the restrictions I have and the fact that there is no VFP client already built (AFAIK), I don't think it will be of great help.
Thanks in advance for your feedback.
Update: The laptop application is a C# WinForm application and is using SQL Server 2008 Express.
The complexity of the situation and requirements leads me to believe you need to write a Visual FoxPro application. Visual FoxPro connects to SQL Server 2008 data easily. The complexity of the code is matching the requirements and identifying the data that needs to be synched, not the syntax. Visual FoxPro strength is in the data manipulation language and the ability to connect to almost any data source (native DBFs, ODBC, ADO, and XML).
SQL Server can read VFP 9 data via the VFP 9 OLE DB driver. You could write T-SQL stored procedures to get to the VFP data. Not sure how it would recognize the laptop being connected to the network though.
Another approach is to use SQL Server XML Diffgrams. I am not an expert by any stretch of the imagination on this approach, but it would be something you can research.
Since my expertise is with Visual FoxPro I would find it way easier to go the other way though, but that is just me. You have to go with the skillset of the resources you have for the project.
VFP reads and writes SQL Server data via a connection (DSN, ConnectionString) and any technique involving SQL Passthrough (SQLConnect(), SQLExec() and SQLDisconnect()), CursorAdapters, Remote Views, or a combination of the three.
A Visual FoxPro program can also recognize Windows Events like connecting to a network. The application could be installed on each laptop and running to recognize the Windows Event. Once the event is raised the application can attempt to connect to the SQL Server database (possible it is connecting to a network without the SQL Server available or a different network).
Once connected it runs the logic to check and synchronize the databases.
Sounds like you don't have a lot of control over the application writing to the VFP 9 data on the laptop. If you do have control over the application writing to the VFP 9 database you might consider changing the app to write to a SQL Server Express instance on the laptop and then you can use SQL Server replication to manage the synchronization. Not a trivial task though and SQL Server replication, while getting better with each release, does cause hair loss in DBAs. Definitely a lot of work going this route.
Rick Schummer
Visual FoxPro MVP
I would encourage you to take another look at MS sync framework. We have a situation where we want to synchronize occasionally connected C# clients apps with our Java/Oracle backend. You can use the sync framework providers for the C# client and implement your own custom subclass of KnowledgeSyncProvider for the backend. This will get you half-way there, and show you a good pattern to apply for the rest.