I'm using Spring Batch to import some old data into my new database. I already exported each table to different csv/excel file.
For instance, account.xlsx has userInfoId,accountName and password.
userinfo.xlsx has id, name, age and contact.
But how to concatenate or read multiple files with mapping to get the foreign key relationship from these records. I need to create new user entity with both info in these two files, what should I do?
I assume the foreign key info is actually IN the files.
Notwithstanding the THRICE DAMNED XML based declaration THIS seems to be what you are trying to do. The key is the MultiResourceItemReader.
Edit: You will almost certainly need an ItemProcessor of course.
Related
I'm trying to write a function to clone a dataset - that is, to create an identical copy to an existing data set, with different primary keys.
I could do this by reading the records, copying the fields one at a time (or using toArray() and fromArray() and unsetting the primary key and resetting any foreign keys along the way), but I was wondering if there's a built-in method for doing this.
I'm using Zend Framework 1.
Nothing similar is implemented out of the box in ZF1. You should create custom Rowset class and define custom clone method which can directly access data of Rowset and apply some transformation e.g. filtering or removal etc.
we have many an application that uses a database that is been used by differrent clients (each client has is own DB). Over the years, some of our client's database has lost some of the foreign key definition.
We would like to use code first Entity Framework, but since not all the db has the relationship defined, we have a lot of problems (specialy if we want to use lazy loading).
We where thinking of trying reverse negeneering a db that has the relations defined and to update only the foreign key definitions, is that possible ?
We only want to fix the foreign key definition and nothing else, because there is critical data in the DB and we don't want to take risks and update the db from the model on the production enveironnement.
Thank you in advance!
I'm having a hard time following your question mainly because I think you have left a lot of information out. I would think that you could reverse engineer the database, then use automatic-migrations to add any foreign keys. YOu may want to look into:
Automatic-Migrations AND Data Annotations or Fluent API
Without more information or an example of your DB Schema or any Code First (CODE) I don't think anyone will be able to do more than point you in the right direction like I have tried to do.
I'm building my EF (v4.0.30319) data model from my SQL Server database. Each table has Created and Updated fields populated via database triggers.
This database is the backend of a ASP.NET Web API application and I recently discovered a problem. The problem is, since the Created and Updated fields are not populated in the model passed into the api endpoint, they are written to the database as NULL. This overwrites any values already in the database for those properties.
I discovered I can edit the EF data model and just delete those two columns from the entity. It works and the datetimes are not overwritten with NULL. But this leads to another, less serious but more annoying, problem... my data model has a bunch of tables that contain these properties and all the tables need to be updated by removing these two columns.
Is there a way to tell EF to ignore certain columns in entities across the entire data model without manually deleting them?
As far as I know, no. Generating the model from the database is going to always create all of the fields from the database table. Well, as long as it has a primary key it will.
It is possible to only update the fields that you want i.e. don't include the "Created" and "Updated" fields in your create and update methods. I'd have to say though, that I'd think it'd be better if those fields didn't even exist on the model at that point. You may at one point see those fields on the model and not remember that they won't get persisted to the DB.
You may want to look into just inserting the datetimes into those fields when you call your create() and update() methods. Then you could just ditch the triggers. You'd obviously want to use a class library for all of your database operations so this functionality would be in one place. To keep it nice and neat, you know?
I have designed a database whose MDF file will be copied to remote offices, so basically I will have different databases wth the same scheme. However, some tables from these databases will have to contain the same data. First I was happy because I knew it was easy to sync them using RowVersion columns in each table, but then I remembered that primary key columns in these tables (columns named "ID") are also identity columns. So I have no idea on how to synchronize them in way that they are identical. With same IDs and everything. Also I am doing this through Entity Framework, which sits between the SQL Server 2008 R2 Express and .NET Framework 4 WCF Service. Any clues?
Note that this is a one-way sync, remote offices need to replicate these tables from the main database but they are not able to modify them and write changes back.
The original thread was started here: http://social.msdn.microsoft.com/Forums/en-US/adodotnetentityframework/thread/e5f89bac-959c-490a-befc-a80d5aa9a9a5/ but I haven't come to a solution yet. If you take a look at the thread I linked to, you will see that the proposed solution was to attach records from the main DB context to the client DB context and call "ApplyCurrentValues" method to update the client DB. However I have come to conclusion that it would not work at all due to these reasons:
Different EntityKey values between data from two contexts. You can't attach a record to a context if that record's EntityKey doesn't correspond with the context. To get past this issue I had to convert the object from mainDB to the object from clientDB using AutoMapper and set the EntityKey manually prior to attaching the record to clientDB context.
If you want to add a new record (if one exists in mainDB but not in clientDB) you can't use Attach. If the record you are trying to attach doesn't exist in the store, EF will throw the exception back at you.
If you want to add a new record, you must use AddObject, but that implies the EntityKey is generated automatically and you will not have control over the identity column. If you try to set EntityKey manually prior to adding a new record, EF will throw an exception at you.
So, the question is, how can I replicate data from the main DB to the client DB using EntityFramework?
We have recently implemented this solution, however our database was simple enough and we had one meta server (we call it meta as it is our server which holds only identities) and we have data servers. We have three data servers doing three way sync. Now originally we only had three servers, but inserting new IDs were problem and we didnt want to use GUID as well because it is not human readable.
So we introduced concept of IdentityServer (we called it MetaServer), which hosts a simple Web Service and simple database, database consists of Tables with only Identities, Hash and LastUpdate, Hash and LastUpdate are used to validate synchronization.
For example, following two tables are there on Meta Server,
Tickets
TicketID (Primary Key,Identity)
LastUpdate (DateTime)
Hash (Hash of Ticket)
Tasks
TaskID (Primary Key, Identity)
LastUpdate (DateTime)
Hash (Hash of Task)
Now Data Servers will contain Tickets as follow,
Tickets
TicketID (Primary Key)
Subject
Message
...
...
Tasks
TaskID (Primary Key)
Subject
Message
...
...
And our Save method on ObjectContext looks like following,
Task task = new Task();
task.TaskID = MetaService.GetNewTaskID();
...
...
// following is save method, checking insert or update, as it is used in
//synchronization, thats why i wrote it like this
void SaveTask(Task task){
Task copy = ObjectContext.Tasks.FirstOrDefault(x=>x.TaskID==task.TaskID);
if(copy==null){
copy = new Task();
ObjectContext.Tasks.AddObject(copy);
}
CloneData(task,copy);
ObjectContext.SaveChanges();
}
To Perform syncing, I would suggest, add a table like this which will Save every Change in Meta Server (Master Server)
Changes
ChangeID
ChangeType = Insert,Update,Delete
ChangeTable
ChangeKey
ChangeTime
Which then every data server can read from Meta Server and update changes...
I see some posts on programmatically mapping a table with a composite key but I haven't found any examples to do that within xml of the edmx file. How do I do this? In the edmx of my entity type I have a key section with multiple propertyref's pointing to my composite key. However, when I go to save my changes it errors and states that there is no update function.
It seems to me, if I mark one key as the primary key and it can manage that it should be able to use a composite key as well.
Please do not reply and state that I shouldn't be using composite keys because I don't care and have no ability to change it. I didn't develop this database as it is a third-party ERP application that I'm interfacing with.
From what I can tell I may have to create a stored procedure just to save it? Isn't there a better way?
Thank You.