I'm using Filehelpers to parse a very wide, fixed format file and want to be able to take the resulting object and load it into a DB using EF. I'm getting a missing key error when I try to load the object into the DB and when I try and add an Id I get a Filehelpers error. So it seems like either fix breaks the other. I know I can map a Filehelpers object to a POCO object and load that but I'm dealing with dozens (sometimes hundreds of columns) so I would rather not have to go through that hassle.
I'm also open to other suggestions for parsing a fixed width file and loading the results into a DB. One option of course is to use an ETL tool but I'd rather do this in code.
Thanks!
This is the FileHelpers class:
public class AccountBalanceDetail
{
[FieldHidden]
public int Id; // Added to try and get EF to work
[FieldFixedLength(1)]
public string RecordNuber;
[FieldFixedLength(3)]
public string Branch;
// Additional fields below
}
And this is the method that's processing the file:
public static bool ProcessFile()
{
var dir = Properties.Settings.Default.DataDirectory;
var engine = new MultiRecordEngine(typeof(AccountBalanceHeader), typeof(AccountBalanceDetail), typeof(AccountBalanceTrailer));
engine.RecordSelector = new RecordTypeSelector(CustomSelector);
var fileName = dir + "\\MOCK_ACCTBAL_L1500.txt";
var res = engine.ReadFile(fileName);
foreach (var rec in res)
{
var type = rec.GetType();
if (type.Name == "AccountBalanceHeader") continue;
if (type.Name == "AccountBalanceTrailer") continue;
var data = rec as AccountBalanceDetail; // Throws an error if AccountBalanceDetail.Id has a getter and setter
using (var ctx = new ApplicationDbContext())
{
// Throws an error if there is no valid Id on AccountBalanceDetail
// EntityType 'AccountBalanceDetail' has no key defined. Define the key for this EntityType.
ctx.AccountBalanceDetails.Add(data);
ctx.SaveChanges();
}
//Console.WriteLine(rec.ToString());
}
return true;
}
Entity Framework needs the key to be a property, not a field, so you could try declaring it instead as:
public int Id {get; set;}
I suspect FileHelpers might well be confused by the autogenerated backing field, so you might need to do it long form in order to be able to mark the backing field with the [FieldHidden] attribute, i.e.,
[FieldHidden]
private int _Id;
public int Id
{
get { return _Id; }
set { _Id = value; }
}
However, you are trying to use the same class for two unrelated purposes and this is generally bad design. On the one hand AccountBalanceDetail is the spec for the import format. On the other you are also trying to use it to describe the Entity. Instead you should create separate classes and map between the two with a LINQ function or a library like AutoMapper.
Related
I have a method that receives an IEnumerable<Guid> of IDs to objects I want to delete. One suggested method is as follows
foreach(Guid id in ids)
{
var tempInstance = new MyEntity { Id = id };
DataContext.Attach(tempInstance); // Exception here
DataContext.Remove(tempInstance);
}
This works fine if the objects aren't already loaded into memory. But my problem is that when they are already loaded then the Attach method throws an InvalidOperationException - The instance of entity type 'MyEntity' cannot be tracked because another instance with the key value 'Id:...' is already being tracked. The same happens if I use DataContext.Remove without calling Attach.
foreach(Guid id in ids)
{
var tempInstance = new MyEntity { Id = id };
DataContext.Remove(tempInstance); // Exception here
}
I don't want to use DataContext.Find to grab the instance of an already loaded object because that will load the object into memory if it isn't already loaded.
I cannot use DataContext.ChangeTracker to find already loaded objects because only objects with modified state appear in there and my objects might be loaded and unmodified.
The following approach throws the same InvalidOperationException when setting EntityEntry.State, even when I override GetHashCode and Equals on MyEntity to ensure dictionary lookups see them as the same object.
foreach(Guid id in ids)
{
var tempInstance = new MyEntity { Id = id };
EntityEntry entry = DataContext.Entry(tempInstance);
entry.State == EntityState.Deleted; // Exception here
}
The only way so far I have found that I can achieve deleting objects by ID without knowing if the object is the following:
foreach(Guid id in ids)
{
var tempInstance = new MyEntity { Id = id };
try
{
DataContext.Attach(tempInstance); // Exception here
}
catch (InvalidOperationException)
{
}
DataContext.Remove(tempInstance);
}
It's odd that I am able to call DataContext.Remove(tempInstance) without error after experiencing an exception trying to Attach it, but at this point it does work without an exception and also deletes the correct rows from the database when DataContext.SaveChanges is executed.
I don't like catching the exception. Is there a "good" way of achieving what I want?
Note: If the class has a self-reference then you need to load the objects into memory so EntityFrameworkCore can determine in which order to delete the objects.
Strangely, although this is a quite common exception in EF6 and EF Core, neither of them expose publicly a method for programmatically detecting the already tracked entity instance with the same key. Note that overriding GetHashCode and Equals doesn't help since EF is using reference equality for tracking entity instances.
Of course it can be obtained from the DbSet<T>.Local property, but it would not be as efficient as the internal EF mechanism used by Find and the methods throwing the aforementioned exception. All we need is the first part of the Find method and returning null when not found instead of loading from the database.
Luckily, for EF Core the method that we need can be implemented relatively easily by using some of the EF Core internals (under the standard This API supports the Entity Framework Core infrastructure and is not intended to be used directly from your code. This API may change or be removed in future releases. policy). Here is the sample implementation, tested on EF Core 2.0.1:
using Microsoft.EntityFrameworkCore.Internal;
namespace Microsoft.EntityFrameworkCore
{
public static partial class CustomExtensions
{
public static TEntity FindTracked<TEntity>(this DbContext context, params object[] keyValues)
where TEntity : class
{
var entityType = context.Model.FindEntityType(typeof(TEntity));
var key = entityType.FindPrimaryKey();
var stateManager = context.GetDependencies().StateManager;
var entry = stateManager.TryGetEntry(key, keyValues);
return entry?.Entity as TEntity;
}
}
}
Now you can use simply:
foreach (var id in ids)
DataContext.Remove(DataContext.FindTracked<MyEntity>(id) ?? new MyEntity { Id = id }));
or
DataContext.RemoveRange(ids.Select(id =>
DataContext.FindTracked<MyEntity>(id) ?? new MyEntity { Id = id }));
I am currently using Mehdi El Gueddari's DbContextScope project, I think by the book, and it's awesome. But I came across a problem I'm unsure how to solve today. I have a query that I need to execute using a different database login/user because it requires additional permissions. I can create another connection string in my web.config, but I'm not sure how to specify that for this query, I want to use this new connection string. Here is my usage:
In my logic layer:
private static IDbContextScopeFactory _dbContextFactory = new DbContextScopeFactory();
public static Guid GetFacilityID(string altID)
{
...
using (_dbContextFactory.CreateReadOnly())
{
entity = entities.GetFacilityID(altID)
}
}
That calls into my data layer which would look something like this:
private AmbientDbContextLocator _dbcLocator = new AmbientDbContextLocator();
protected CRMEntities DBContext
{
get
{
var dbContext = _dbcLocator.Get<CRMEntities>();
if (dbContext == null)
throw new InvalidOperationException("No ambient DbContext....");
return dbContext;
}
}
public virtual Guid GetFaciltyID(string altID)
{
return DBContext.Set<Facility>().Where(f => f.altID = altID).Select(f => f.ID).FirstOrDefault();
}
Currently my connection string is set in the default way:
public partial class CRMEntities : DbContext
{
public CRMEntities()
: base("name=CRMEntities")
{}
}
Is it possible for this specific query to use a different connection string and how?
I ended up modifying the source code in a way that feels slightly hacky, but is getting the job done for now. I created a new IAmbientDbContextLocator with a Get<TDbContext> method override that accepts a connection string:
public TDbContext Get<TDbContext>(string nameOrConnectionString) where TDbContext : DbContext
{
var ambientDbContextScope = DbContextScope.GetAmbientScope();
return ambientDbContextScope == null ? null : ambientDbContextScope.DbContexts.Get<TDbContext>(nameOrConnectionString);
}
Then I updated the DbContextCollection to pass this parameter to the DbContext's existing constructor overload. Last, I updated the DbContextCollection maintain a Dictionary<KeyValuePair<Type, string>, DbContext> instead of a Dictionary<Type, DbContext> as its cached _initializedDbContexts where the added string is the nameOrConnectionString param. So in other words, I updated it to cache unique DbContext type/connection string pairs.
Then I can get at the DbContext with the connection I need like this:
var dbContext = new CustomAmbientDbContextLocator().Get<CRMEntities>("name=CRMEntitiesAdmin");
Of course you'd have to be careful your code doesn't end up going through two different contexts/connection strings when it should be going through the same one. In my case I have them separated into two different data access class implementations.
When a new Post is input into the system, a number of Tags will need to be instantiated and associated with the Post instance. Some of these Tags will already exist in the db, others will not and will need inserting.
An example:
var post = new Post {
Slug = "hello-world",
Title = "Hello, World!",
Content = "this is my first post.",
Tags = new List<Tag>()
};
var tag = new Tag { Name = "introduction" };
post.Tags.Add(tag);
When an associated Tag does not exist in the db, I can rely on a simple call to DbSet<T>.Add to insert both the post and the associated tags into the db.
However, attempting to insert a post with associated tags that already exist in the db causes a primary key violation on the tags table.
In an attempt to solve this problem, I tried to Attach each tag to the database context which works super when the tags already exist in the database but otherwise, an exception is thrown with the following inner exception:
The INSERT statement conflicted with the FOREIGN KEY constraint "FK_dbo.TagPosts_dbo.Tags_Tag_Name". The conflict occurred in
database "EF.Domain.BlogDb", table "dbo.Tags", column
'Name'. The statement has been terminated."}
I want to insert tags associated with the post into the database only when it is required. How can I achieve this?
Your predicament is quite interesting. If you wouldn't use a generic repository but a specific repository that knows the primary key of the entity you want to attach is you could just use a code like this:
var tagExists = Tags.Any(t => t.Name == tag.Name);
or
var tag = Tags.Find(tag.Name);
however in your case we need a more general approach such as getting the primary key of the entity the Repository class uses regardless the type of the entity. To achieve this I've created two extension methods on the DbContext class:
public static IList<string> GetPrimaryKeyNames<TEntity>(this DbContext context)
where TEntity : class
{
var objectContext = ((IObjectContextAdapter)context).ObjectContext;
var set = objectContext.CreateObjectSet<TEntity>();
return set.EntitySet.ElementType
.KeyMembers
.Select(k => k.Name)
.ToList();
}
public static IList<object> GetPrimaryKeyValues<TEntity>(this DbContext context, TEntity entity)
where TEntity : class
{
var valueList = new List<object>();
var primaryKeyNames = context.GetPrimaryKeyNames<TEntity>();
foreach(var propertyInfo in entity.GetType().GetProperties())
{
if (primaryKeyNames.Contains(propertyInfo.Name))
{
valueList.Add(propertyInfo.GetValue(entity));
}
}
return valueList;
}
Utilizing these methods you can change the Attach method on the Repository class like the following:
public void Attach(TEntity entity)
{
var storeEntity = _context.Set<TEntity>().Find(
_context.GetPrimaryKeyValues(entity).ToArray());
if (storeEntity != null)
{
_context.Entry(storeEntity).State = EntityState.Detached;
_context.Set<TEntity>().Attach(entity);
}
}
I'm a little confused as to the purpose of a data model in Entity Framework code-first. Because EF will auto-generate a database from scratch for you if it doesn't already exist using nothing more than the data model (including data annotations and Fluent API stuff in DbContext.OnModelCreating), I was assuming that the data model should fully describe your database's structure, and you wouldn't need to modify anything fundamental after that.
However, I came across this Codeplex issue in which one of the EF Triage Team members suggests that custom indexes be added in data migrations, but not as annotations to your data model fields, or Fluent API code.
But wouldn't that mean that anyone auto-generating the database from scratch would not get those custom indexes added to their DB? The assumption seems to be that once you start using data migrations, you're never going to create the database from scratch again. What if you're working in a team and a new team member comes along with a new SQL Server install? Are you expected to copy over a database from another team member? What if you want to start using a new DBMS, like Postgres? I thought one of the cool things about EF was that it was DBMS-independent, but if you're no longer able to create the database from scratch, you can no longer do things in a DBMS-independent way.
For the reasons I outlined above, wouldn't adding custom indexes in a data migration but not in the data model be a bad idea? For that matter, wouldn't adding any DB structure changes in a migration but not in the data model be a bad idea?
Are EF code-first models intended to fully describe a database's structure?
No, they don't fully describe the database structure or schema.Still there are methods to make the database fully described using EF. They are as below:
You can use the new CTP5’s ExecuteSqlCommand method on Database class which allows raw SQL commands to be executed against the database.
The best place to invoke SqlCommand method for this purpose is inside a Seed method that has been overridden in a custom Initializer class. For example:
protected override void Seed(EntityMappingContext context)
{
context.Database.ExecuteSqlCommand("CREATE INDEX IX_NAME ON ...");
}
You can even add Unique Constraints this way.
It is not a workaround, but will be enforced as the database will be generated.
OR
If you are badly in need of the attribute, then here it goes
[AttributeUsage(AttributeTargets.Property, Inherited = false, AllowMultiple = true)]
public class IndexAttribute : Attribute
{
public IndexAttribute(string name, bool unique = false)
{
this.Name = name;
this.IsUnique = unique;
}
public string Name { get; private set; }
public bool IsUnique { get; private set; }
}
After this , you will have an initializer which you will call in your OnModelCreating method as below:
public class IndexInitializer<T> : IDatabaseInitializer<T> where T : DbContext
{
private const string CreateIndexQueryTemplate = "CREATE {unique} INDEX {indexName} ON {tableName} ({columnName});";
public void InitializeDatabase(T context)
{
const BindingFlags PublicInstance = BindingFlags.Public | BindingFlags.Instance;
Dictionary<IndexAttribute, List<string>> indexes = new Dictionary<IndexAttribute, List<string>>();
string query = string.Empty;
foreach (var dataSetProperty in typeof(T).GetProperties(PublicInstance).Where(p => p.PropertyType.Name == typeof(DbSet<>).Name))
{
var entityType = dataSetProperty.PropertyType.GetGenericArguments().Single();
TableAttribute[] tableAttributes = (TableAttribute[])entityType.GetCustomAttributes(typeof(TableAttribute), false);
indexes.Clear();
string tableName = tableAttributes.Length != 0 ? tableAttributes[0].Name : dataSetProperty.Name;
foreach (PropertyInfo property in entityType.GetProperties(PublicInstance))
{
IndexAttribute[] indexAttributes = (IndexAttribute[])property.GetCustomAttributes(typeof(IndexAttribute), false);
NotMappedAttribute[] notMappedAttributes = (NotMappedAttribute[])property.GetCustomAttributes(typeof(NotMappedAttribute), false);
if (indexAttributes.Length > 0 && notMappedAttributes.Length == 0)
{
ColumnAttribute[] columnAttributes = (ColumnAttribute[])property.GetCustomAttributes(typeof(ColumnAttribute), false);
foreach (IndexAttribute indexAttribute in indexAttributes)
{
if (!indexes.ContainsKey(indexAttribute))
{
indexes.Add(indexAttribute, new List<string>());
}
if (property.PropertyType.IsValueType || property.PropertyType == typeof(string))
{
string columnName = columnAttributes.Length != 0 ? columnAttributes[0].Name : property.Name;
indexes[indexAttribute].Add(columnName);
}
else
{
indexes[indexAttribute].Add(property.PropertyType.Name + "_" + GetKeyName(property.PropertyType));
}
}
}
}
foreach (IndexAttribute indexAttribute in indexes.Keys)
{
query += CreateIndexQueryTemplate.Replace("{indexName}", indexAttribute.Name)
.Replace("{tableName}", tableName)
.Replace("{columnName}", string.Join(", ", indexes[indexAttribute].ToArray()))
.Replace("{unique}", indexAttribute.IsUnique ? "UNIQUE" : string.Empty);
}
}
if (context.Database.CreateIfNotExists())
{
context.Database.ExecuteSqlCommand(query);
}
}
private string GetKeyName(Type type)
{
PropertyInfo[] propertyInfos = type.GetProperties(BindingFlags.FlattenHierarchy | BindingFlags.Instance | BindingFlags.Public);
foreach (PropertyInfo propertyInfo in propertyInfos)
{
if (propertyInfo.GetCustomAttribute(typeof(KeyAttribute), true) != null)
return propertyInfo.Name;
}
throw new Exception("No property was found with the attribute Key");
}
}
Then overload OnModelCreating in your DbContext
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
Database.SetInitializer(new IndexInitializer<MyContext>());
base.OnModelCreating(modelBuilder);
}
Apply the index attribute to your Entity type, with this solution you can have multiple fields in the same index just use the same name and unique.
OR
You can do the migrations later on.
Note:
I have taken a lot of this code from here.
The question seems to be if there is value in having migrations added mid-stream, or if those will cause problems for future database initializations on different machines.
The initial migration that is created also contains the entire data model as it exists, so by adding migrations (enable-migrations in the Package Manager Console) you are, in effect, creating the built-in mechanism for your database to be properly created down the road for other developers.
If you're doing this, I do recommend modifying the database initialization strategy to run all your existing migrations, lest EF should start up and get the next dev's database out of sync.
Something like this would work:
Database.SetInitializer(new MigrateDatabaseToLatestVersion<YourNamespace.YourDataContext, Migrations.Configuration>());
So, no, this won't inherently introduce problems for future work/developers. Remember that migrations are just turned into valid SQL that executes against the database...you can even use script mode to output the TSQL required to make the DB modifications based on anything in the migrations you have created.
Cheers.
How can I update a single property of a record without retrieving it first?
I'm asking in the context of EF Code First 4.1
Says I have a class User, mapping to table Users in Database:
class User
{
public int Id {get;set;}
[Required]
public string Name {get;set;}
public DateTime LastActivity {get;set;}
...
}
Now I want to update LastActivity of a user. I have user id. I can easily do so by querying the user record, set new value to LastActivity, then call SaveChanges(). But this would result in a redundant query.
I work around by using Attach method. But because EF throws a validation exception on Name if it's null, I set Name to a random string (will not be updated back to DB). But this doesn't seem a elegant solution:
using (var entities = new MyEntities())
{
User u = new User {Id = id, Name="this wont be updated" };
entities.Users.Attach(u);
u.LastActivity = DateTime.Now;
entities.SaveChanges();
}
I would be very appriciate if someone can provide me a better solution. And forgive me for any mistake as this is the first time I've asked a question on SO.
This is a problem of validation implementation. The validation is able to validate only a whole entity. It doesn't validate only modified properties as expected. Because of that the validation should be turned off in scenarios where you want to use incomplete dummy objects:
using (var entities = new MyEntities())
{
entities.Configuration.ValidateOnSaveEnabled = false;
User u = new User {Id = id, LastActivity = DateTime.Now };
entities.Users.Attach(u);
entities.Entry(user).Property(u => u.LastActivity).IsModified = true;
entities.SaveChanges();
}
This is obviously a problem if you want to use the same context for update of dummy objects and for update of whole entities where the validation should be used. The validation take place in SaveChanges so you can't say which objects should be validated and which don't.
I'm actually dealing with this right now. What I decided to do was override the ValidateEntity method in the DB context.
protected override DbEntityValidationResult ValidateEntity(DbEntityEntry entityEntry, IDictionary<object, object> items)
{
var result = base.ValidateEntity(entityEntry, items);
var errors = new List<DbValidationError>();
foreach (var error in result.ValidationErrors)
{
if (entityEntry.Property(error.PropertyName).IsModified)
{
errors.Add(error);
}
}
return new DbEntityValidationResult(entityEntry, errors);
}
I'm sure there's some holes that can be poked in it, but it seemed better than the alternatives.
You can try a sort of hack:
context.Database.ExecuteSqlCommand("update [dbo].[Users] set [LastActivity] = #p1 where [Id] = #p2",
new System.Data.SqlClient.SqlParameter("p1", DateTime.Now),
new System.Data.SqlClient.SqlParameter("p2", id));