Synchronizing data with identity columns between SQL Express Servers using Entity Framework - entity-framework

I have designed a database whose MDF file will be copied to remote offices, so basically I will have different databases wth the same scheme. However, some tables from these databases will have to contain the same data. First I was happy because I knew it was easy to sync them using RowVersion columns in each table, but then I remembered that primary key columns in these tables (columns named "ID") are also identity columns. So I have no idea on how to synchronize them in way that they are identical. With same IDs and everything. Also I am doing this through Entity Framework, which sits between the SQL Server 2008 R2 Express and .NET Framework 4 WCF Service. Any clues?
Note that this is a one-way sync, remote offices need to replicate these tables from the main database but they are not able to modify them and write changes back.
The original thread was started here: http://social.msdn.microsoft.com/Forums/en-US/adodotnetentityframework/thread/e5f89bac-959c-490a-befc-a80d5aa9a9a5/ but I haven't come to a solution yet. If you take a look at the thread I linked to, you will see that the proposed solution was to attach records from the main DB context to the client DB context and call "ApplyCurrentValues" method to update the client DB. However I have come to conclusion that it would not work at all due to these reasons:
Different EntityKey values between data from two contexts. You can't attach a record to a context if that record's EntityKey doesn't correspond with the context. To get past this issue I had to convert the object from mainDB to the object from clientDB using AutoMapper and set the EntityKey manually prior to attaching the record to clientDB context.
If you want to add a new record (if one exists in mainDB but not in clientDB) you can't use Attach. If the record you are trying to attach doesn't exist in the store, EF will throw the exception back at you.
If you want to add a new record, you must use AddObject, but that implies the EntityKey is generated automatically and you will not have control over the identity column. If you try to set EntityKey manually prior to adding a new record, EF will throw an exception at you.
So, the question is, how can I replicate data from the main DB to the client DB using EntityFramework?

We have recently implemented this solution, however our database was simple enough and we had one meta server (we call it meta as it is our server which holds only identities) and we have data servers. We have three data servers doing three way sync. Now originally we only had three servers, but inserting new IDs were problem and we didnt want to use GUID as well because it is not human readable.
So we introduced concept of IdentityServer (we called it MetaServer), which hosts a simple Web Service and simple database, database consists of Tables with only Identities, Hash and LastUpdate, Hash and LastUpdate are used to validate synchronization.
For example, following two tables are there on Meta Server,
Tickets
TicketID (Primary Key,Identity)
LastUpdate (DateTime)
Hash (Hash of Ticket)
Tasks
TaskID (Primary Key, Identity)
LastUpdate (DateTime)
Hash (Hash of Task)
Now Data Servers will contain Tickets as follow,
Tickets
TicketID (Primary Key)
Subject
Message
...
...
Tasks
TaskID (Primary Key)
Subject
Message
...
...
And our Save method on ObjectContext looks like following,
Task task = new Task();
task.TaskID = MetaService.GetNewTaskID();
...
...
// following is save method, checking insert or update, as it is used in
//synchronization, thats why i wrote it like this
void SaveTask(Task task){
Task copy = ObjectContext.Tasks.FirstOrDefault(x=>x.TaskID==task.TaskID);
if(copy==null){
copy = new Task();
ObjectContext.Tasks.AddObject(copy);
}
CloneData(task,copy);
ObjectContext.SaveChanges();
}
To Perform syncing, I would suggest, add a table like this which will Save every Change in Meta Server (Master Server)
Changes
ChangeID
ChangeType = Insert,Update,Delete
ChangeTable
ChangeKey
ChangeTime
Which then every data server can read from Meta Server and update changes...

Related

Spring Data JDBC: Can I create my UUID PKs on the client side, and not on the server? [duplicate]

I'm playing around with spring-data-jdbc and discovered a problem, with I can't solve using Google.
No matter what I try to do, I just can't push a trivial object into the database (Bean1.java:25):
carRepository.save(new Car(2L, "BMW", "5"));
Both, without one and with a TransactionManager +#Transactional the database (apparently) does not commit the record.
The code is based on a Postgres database, but you might also simply use a H2 below and get the same result.
Here is the (minimalistic) source code:
https://github.com/bitmagier/spring-data-jdbc-sandbox/tree/stackoverflow-question
Can somebody tell me, why the car is not inserted into the database?
This is not related to transactions not working.
Instead, it's about Spring Data JDBC considering your instance an existing instance that needs updating (instead of inserting).
You can verify this is the problem by activating logging for org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate. You should see an update but no insert.
By default, Spring Data JDBC considers an entity as new when it has an id of an object type and a value of null or of a primitive type (e.g. int or long) and a value of 0.
If your entity has an attribute with #Version annotation that attribute will be used to determine if the instance is a new one.
You have the following options in order to make it work:
Set the id to null and configure your database schema so that it will automatically create a new value on insert. After the save your entity instance will contain the generated value from the database.
Note: Spring Data JDBC will set the id even if it is final in your entity.
Leave the id null and set it in a Before-Convert listener to the desired value.
Let your entity implement Persistable. This allows you to control when an entity is considered new. You'll probably need a listener as well so you can let the entity know it is not new any longer.
Beginning with version 1.1 of Spring Data JDBC you'll also be able to use a JdbcAggregateTemplate to do a direct insert, without inspecting the id, see https://jira.spring.io/browse/DATAJDBC-282. Of course, you can do that in a custom method of your repository, as is done in this example: https://github.com/spring-projects/spring-data-examples/pull/441

entityManager.merge(object) is persisting only the last record from list

I have two records in my ArrayList<Object> but when I'm trying to insert these records in database, only the last record gets saved into DB.
I'm using Spring data JPA entityManager.merge() method since I have custom sequenceGenerator to be used for Entities.
summaryList.stream().forEach{(summary -> entityManager.merge(summary)};
On debugging I get two records in summaryList but when I check my DB table, only one record gets inserted.
Hibernate generates a SELECT statement first to fetch the latest state of the underlying database record, and then, it copies the detached entity state onto the newly fetched managed entity. This way, the dirty checking mechanism can detect any state change and propagate it to the database.
While for IDENTITY and SEQUENCE generator strategies, you can practically use merge to persist an entity, for the assigned generator, this would be less efficient.
You can actually fix this issue by adding a version property to your entity.
#Version
private Long version;

Trouble with Multi-Tenant Schema Generator Example

We are attempting to use CFE to generate one schema for each tenant as outlined in the CodeFluent blog post (http://blog.codefluententities.com/2014/12/04/multi-tenant-using-multiple-schema/). In this scenario, we are expecting that each schema generated should be identical and we are using the ICodeFluentPersistence Hook system to identify the company for a user and then properly set the schema to be used. All of that works fine, but when we run the code to generate the multiple schemas (https://github.com/SoftFluent/CodeFluent-Entities/tree/master/Extensions/SoftFluent.MultiTenantGenerator), it is removing the constraints. I then tried to see if there was an issue with my configuration, but running the sample program from GitHub produces the same results. After running the sample program, the Primary key was not present in the contoso schema, even though is was properly defined in the dbo schema (and in the model).
Has anyone used the CFE Multi-Schema generator or have any insight into what the issue may be?
Thanks for your response, but I am not sure that I agree. The whole reason (at least of me) to use the Multi-Tenant generator is to create as many database schemas as needed (one per client) from a single CFE model. The idea that you would lose the constraints in all but one of them didn't feel right so I did a bit more investigation and found the following in "Microsoft SQL Server 2012 Internals" by Kalen Delaney and Craig Freeman (through Google Books):
And in fact was able to do a quick test to prove this out by creating two identical tables with identical PK names:
So it would appear to me that CFE should be able to create the two identical databases from the same model and seems to point to a deficiency in the SQLServer diff engine.
The multi-schema generator loads the model and change it dynamically to modify the schema of the entities. Then it call the standard code production process with only the database producers (SQL Server, Oracle, etc.).
So if you want to generate 2 differents schema (dbo and contoso) against an empty database, the process is the following:
Generate the database for the dbo schema from a blank database
Generate the database for the contoso schema from the previously generated database
Before creating a constraint, the SQL Server diff engine drops the constraint with the same name. In fact SQL Server does not allow 2 constraints to have the same name (I can't find a page on MSDN with more details about that). So in your case the existing PK is dropped when you generate the contoso schema because the name of the PK is the same as the one that exists in the dbo schema. Maybe this can be improved, but the diffs engine tries to generate a code that works for SQL Server 2000 to SQL Server 2016.
Workarounds
You can generate each schema in a different database, so the diffs engine will generate the code you expect. Then you can run the generated scripts on the production database. Not the easiest way but it should work.
You can use the patch producer to replace the name of the schema in the file. For SQL files you should use the SqlServerPatchProducer as explain in the KnowledgeBase:
namespace Sample
{
public class SqlServerPatchProducer : SqlServerProducer
{
public SqlServerPatchProducer()
{
}
protected override void RunProceduresScript()
{
string path = GetPath(Project.DefaultNamespace + "_procedures.sql");
ProduceFrom(path, "before");
SearchAndReplaceProducer.ProducePatches(Project, null, this, null, ProductionFlags, Element);
Utilities.RunFileScript(path, Database, OutputEncoding);
ProduceFrom(path, "after");
}
}
}

Entity Framework 5: RelationShips and Attached/Detached objects

I'm a bit confused how to work properly with many-to-many relationships in locally created objects.
I've prepared a small example to demonstrate the issue.
There are two tables and one mapping table:
And data in tables:
Entity Framework has created two tables and many-to-many relation:
Now run the application and load single student (with ID=1 and his classrooms).
If student was loaded from the database then everything is fine and EF gets 2 classrooms:
But in case user was created locally (with the same Id but changed Name) and attached to the EF then classrooms are not loaded (studentLocal variable).
The same situation if I try to load student from EF - it gets the local user (student variable has Name="xx") and no classrooms:
On the view user can change classrooms for student as well as change student properties so I need to update Student table and merge StudentClassroom table.
What is the best way to deal in this case? I don't want to load each
postback student entity again and again.
Is there a way to load
Classrooms for locally created Student which exist in the database?
How to merge StudentClassroom records? I know only one - load
existed records and new one and merge then either deleting all of
them and recreating from the new list either manually determining
which records should be deleted/updated/created. Is there a better
approach?
Your problem is that EF is not creating the right tables for your model. It should be creating 3 classes not 2. Probably the reason it isn't is that you don't have the relations set up properly in the database. If you are using SQL Server try using the diagram feature to check your relationships and Primary Keys are correctly set up before you set up the model.

Create new or update existing entity at one go with JPA

A have a JPA entity that has timestamp field and is distinguished by a complex identifier field. What I need is to update timestamp in an entity that has already been stored, otherwise create and store new entity with the current timestamp.
As it turns out the task is not as simple as it seems from the first sight. The problem is that in concurrent environment I get nasty "Unique index or primary key violation" exception. Here's my code:
// Load existing entity, if any.
Entity e = entityManager.find(Entity.class, id);
if (e == null) {
// Could not find entity with the specified id in the database, so create new one.
e = entityManager.merge(new Entity(id));
}
// Set current time...
e.setTimestamp(new Date());
// ...and finally save entity.
entityManager.flush();
Please note that in this example entity identifier is not generated on insert, it is known in advance.
When two or more of threads run this block of code in parallel, they may simultaneously get null from entityManager.find(Entity.class, id) method call, so they will attempt to save two or more entities at the same time, with the same identifier resulting in error.
I think that there are few solutions to the problem.
Sure I could synchronize this code block with a global lock to prevent concurrent access to the database, but would it be the most efficient way?
Some databases support very handy MERGE statement that updates existing or creates new row if none exists. But I doubt that OpenJPA (JPA implementation of my choice) supports it.
Event if JPA does not support SQL MERGE, I can always fall back to plain old JDBC and do whatever I want with the database. But I don't want to leave comfortable API and mess with hairy JDBC+SQL combination.
There is a magic trick to fix it using standard JPA API only, but I don't know it yet.
Please help.
You are referring to the transaction isolation of JPA transactions. I.e. what is the behaviour of transactions when they access other transactions' resources.
According to this article:
READ_COMMITTED is the expected default Transaction Isolation level for using [..] EJB3 JPA
This means that - yes, you will have problems with the above code.
But JPA doesn't support custom isolation levels.
This thread discusses the topic more extensively. Depending on whether you use Spring or EJB, I think you can make use of the proper transaction strategy.