Debugging Code Called by EF Core Add-Migrations - entity-framework-core

I have an Entity Framework Core database defined in a separate assembly, using the IDesignTimeDbContextFactory<> pattern (i.e., I define a class, derived from IDesignTimeDbContextFactory, which has a method called CreateDbContext that returns an instance of the database context).
Because the application of which the EF Core database is a part utilizes AutoFac dependency injection, the IDesignTimeDbContextFactory<> factory class creates an AutoFac container in its constructor, and then resolves the DbContextOptionsBuilder<>-derived class which is fed into the constructor for the database context (I do this so I can control whether a local or an Azure-based SqlServer database is targeted, based on a config file setting, with passwords stored in an Azure KeyVault):
public class TemporaryDbContextFactory : IDesignTimeDbContextFactory<FitchTrustContext>
{
private readonly FitchTrustDBOptions _dbOptions;
public TemporaryDbContextFactory()
{
// OMG, I would >>never<< have thought to do this to eliminate the default logging by this
// deeply-buried package. Thanx to Bruce Chen via
// https://stackoverflow.com/questions/47982194/suppressing-console-logging-by-azure-keyvault/48016958#48016958
LoggerCallbackHandler.UseDefaultLogging = false;
var builder = new ContainerBuilder();
builder.RegisterModule<SerilogModule>();
builder.RegisterModule<KeyVaultModule>();
builder.RegisterModule<ConfigurationModule>();
builder.RegisterModule<FitchTrustDbModule>();
var container = builder.Build();
_dbOptions = container.Resolve<FitchTrustDBOptions>() ??
throw new NullReferenceException(
$"Could not resolve {typeof(FitchTrustDBOptions).Name}");
}
public FitchTrustContext CreateDbContext( string[] args )
{
return new FitchTrustContext( _dbOptions );
}
}
public class FitchTrustDBOptions : DbContextOptionsBuilder<FitchTrustContext>
{
public FitchTrustDBOptions(IFitchTrustNGConfigurationFactory configFactory, IKeyVaultManager kvMgr)
{
if (configFactory == null)
throw new NullReferenceException(nameof(configFactory));
if (kvMgr == null)
throw new NullReferenceException(nameof(kvMgr));
var scannerConfig = configFactory.GetFromDisk()
?? throw new NullReferenceException(
"Could not retrieve ScannerConfiguration from disk");
var dbConnection = scannerConfig.Database.Connections
.SingleOrDefault(c =>
c.Location.Equals(scannerConfig.Database.Location,
StringComparison.OrdinalIgnoreCase))
?? throw new ArgumentOutOfRangeException(
$"Cannot find database connection information for location '{scannerConfig.Database.Location}'");
var temp = kvMgr.GetSecret($"DatabaseCredentials--{dbConnection.Location}--Password");
var connString = String.IsNullOrEmpty(dbConnection.UserID) || String.IsNullOrEmpty(temp)
? dbConnection.ConnectionString
: $"{dbConnection.ConnectionString}; User ID={dbConnection.UserID}; Password={temp}";
this.UseSqlServer(connString,
optionsBuilder =>
optionsBuilder.MigrationsAssembly(typeof(FitchTrustContext).GetTypeInfo().Assembly.GetName()
.Name));
}
}
Needless to say, while this provides me with a lot of flexibility (I can switch from local to cloud database just by changing a single config parameter, and any required passwords are reasonably securely stored in the cloud), it can trip up the add-migration commandlet if there's a bug in the code (e.g., the wrong name of a configuration file).
To debug those kinds of problems, I've often had to resort to outputting messages to the Visual Studio output window via diagnostic WriteLine calls. That strikes me as pretty primitive (not to mention time-consuming).
Is there a way to attach a debugger to my code that's called by add-migration so I can step thru it, check values, etc? I tried inserting a Launch() debugger line in my code, but it doesn't work. It seems to throw me into add-manager codebase, for which I have no symbols loaded, and breakpoints that I try to set in my code show up as the empty red circle: they'll never be hit.
Thoughts and suggestions would be most welcome!

Add Debugger.Launch() to the beginning of the constructor to launch the just-in-time debugger. This lets you attach VS to the process and debug it like normal.

Related

Using EF with stored procedure and getting error as New transaction is not allowed because there are other threads running in the session

In Entity framework, I have implemented generic repository and unit of work pattern.
Below is related stuff from unit of work:
public IRepository<TEntity, TKey> GetRepository<TEntity, TKey>() where TEntity : class
{
if (_repositories == null)
{
_repositories = new Dictionary<string, object>();
}
string key = String.Format("{0}|{1}", typeof(TEntity).Name, typeof(TKey).Name);
if (_repositories.ContainsKey(key))
{
return (IRepository<TEntity, TKey>)_repositories[key];
}
Type repositoryType = typeof(Repository<TEntity, TKey>);
_repositories.Add(key, Activator.CreateInstance(repositoryType, _dataContext));
return (IRepository<TEntity, TKey>)_repositories[key];
}
From Manager layer, entity framework is call as below:
IRepository<tablenameEntity int> _tableEntityRepository = _unitOfWork.GetRepository<tablenameEntity, int>();
Error is as follow:
An error occurred while starting a transaction on the provider connection. See the inner exception for details.
{"New transaction is not allowed because there are other threads running in the session."}.
Actually, I just did as below,
remove stored procedure and function from model browser and added procedure and function again.
run custom tool on edmx file.
It just work out.
I am not sure, what was the issue. did that before also, but it was not work.. now, it works and not produce after that.
Thanks

what is a good way to get test data to a mock repo

I have a database (SQL Server, Entity Framework) with 10-12 tables, around 5000 records. I want to code a unit test for a complex report that uses the data from those tables. What would be a good way to do that? I really do not want unit tests hitting an actual database...
Late to answer but its worth pointing out the NuGet package Effort allows you to fake an entire DbContext in memory from either a code-first or a database-first model.
I use it frequently & like it's ease of set up. You can use it to test issues that one could miss if trying to unit test with concrete fakes such as List<Entity>. (Issues caused by trying to load nav properties after the context is disposed for example)
A downside is that it does slow down the unit test a little1 which may be a concern for TDD or with a huge number of tests but I have always found it perfectly acceptable.
Simple use case outline for a db first set up would be as follows:
Add the nuget package for your EF version (package id: Effort.EF6 perhaps?)
Add an override to the constructor of your "MyEntities" class to enable Effort to pass in it's own connection string:
public partial class MyEntities
{
public MyEntities(DbConnection connection) : base(connection, true)
{
}
}
You can now create a 'Persistent' db store (persistent means it stays available after the context is disposed - it's not saved anywhere after the test run finishes!) with the following code:
var context = new MyEntities(Effort.EntityConnectionFactory.CreatePersistent("name=MyEntities"))
Just make sure you have the connection string to the db available in your app.config for the test project (used only to create the model)
Set up whatever mock data you need to test the context (I often have some simple data loaded using [ClassInitialize] or [TestInitialize] methods but it's easy to do whatever makes sense for your own set up):
[ClassInitialize]
public static void Initialise()
{
using (var context = new MyEntities(Effort.EntityConnectionFactory.CreatePersistent("name=MyEntities")))
{
context.Employees.Add(new Employee { Name = "John Doe", Skills = null});
context.Skills.Add(new Skill { Type = "C# Coding"});
context.SaveChanges();
}
}
Run your unit test using the in memory context:
[TestMethod]
public async Task CsSkillCanBeAssociatedWithEmployee()
{
//Arrange
using (var context = new MyEntities(Effort.EntityConnectionFactory.CreatePersistent("name=MyEntities")))
using (var sut = new SkillsMatrixService(context))
{
//Act
await sut.AddSkillToEmployeeAsync("Joe Bloggs","C# Coding");
}
//Assert
using (var context = new MyEntities(Effort.EntityConnectionFactory.CreatePersistent("name=MyEntities")))
{
var joesCSharpSkills = context
.Employees
.First(e => e.Name == "Joe Bloggs")
.Skills
.Where(s => s.Type == "C# Coding");
Assert.IsTrue(joesCSharpSkills.Any());
}
}
I'm given to understand you can load the initial set up for your db from csv config files to live in your project but I've always just set up dummy data in code as above.
1. Anecdotally, in my own (fairly limited) experience, 1 second or so is about usual to set it up and load some dummy data for a smallish model with around 20 entities and lots of nav properties - my machine isn't ultra fast. Naturally, your mileage may vary according to the complexity of your model and your hardware setup

Entity Framework MigrateDatabaseToLatestVersion giving error

I am attempting to use Entity Framework code based migrations with my web site. I currently have a solution with multiple projects in it. There is a Web API project which I want to initialize the database and another project called the DataLayer project. I have enabled migrations in the DataLayer project and created an initial migration that I am hoping will be used to create the database if it does not exist.
Here is the configuration I got when I enabled migrations
public sealed class Configuration : DbMigrationsConfiguration<Harris.ResidentPortal.DataLayer.ResidentPortalContext>
{
public Configuration()
{
AutomaticMigrationsEnabled = false;
}
protected override void Seed(Harris.ResidentPortal.DataLayer.ResidentPortalContext context)
{
// This method will be called after migrating to the latest version.
// You can use the DbSet<T>.AddOrUpdate() helper extension method
// to avoid creating duplicate seed data. E.g.
//
// context.People.AddOrUpdate(
// p => p.FullName,
// new Person { FullName = "Andrew Peters" },
// new Person { FullName = "Brice Lambson" },
// new Person { FullName = "Rowan Miller" }
// );
//
}
}
The only change I made to this after it was created was to change it from internal to public so the WebAPI could see it and use it in it's databaseinitializer. Below is the code in the code in the Application_Start that I am using to try to initialize the database
Database.SetInitializer(new MigrateDatabaseToLatestVersion<ResidentPortalContext, Configuration>());
new ResidentPortalUnitOfWork().Context.Users.ToList();
If I run this whether or not a database exists I get the following error
Directory lookup for the file "C:\Users\Dave\Documents\Visual Studio 2012\Projects\ResidentPortal\Harris.ResidentPortal.WebApi\App_Data\Harris.ResidentPortal.DataLayer.ResidentPortalContext.mdf" failed with the operating system error 2(The system cannot find the file specified.).
CREATE DATABASE failed. Some file names listed could not be created. Check related errors.
It seems like it is looking in the totally wrong place for the database. It seems to have something to do with this particular way I am initializing the database because if I change the code to the following.
Database.SetInitializer(new DropCreateDatabaseAlways<ResidentPortalContext>());
new ResidentPortalUnitOfWork().Context.Users.ToList();
The database will get correctly created where it needs to go.
I am at a loss for what is causing it. Could it be that I need to add something else to the configuration class or does it have to do with the fact that all my migration information is in the DataLayer project but I am calling this from the WebAPI project?
I have figured out how to create a dynamic connection string for this process. You need to first add this line into your EntityFramework entry on Web or App.Config instead of the line that gets put there by default.
<defaultConnectionFactory type="<Namespace>.<ConnectionStringFacotry>, <Assembly>"/>
This tells the program you have your own factory that will return a DbConnection. Below is the code I used to make my own factory. Part of this is a hack to get by the fact that a bunch of programmers work on the same set of code but some of us use SQL Express while others use full blown SQL Server. But this will give you an example to go by for what you need.
public sealed class ResidentPortalConnectionStringFactory: IDbConnectionFactory
{
public DbConnection CreateConnection(string nameOrConnectionString)
{
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder(ConfigurationManager.ConnectionStrings["PortalDatabase"].ConnectionString);
//save off the original catalog
string originalCatalog = builder.InitialCatalog;
//we're going to connect to the master db in case the database doesn't exist yet
builder.InitialCatalog = "master";
string masterConnectionString = builder.ToString();
//attempt to connect to the master db on the source specified in the config file
using (SqlConnection conn = new SqlConnection(masterConnectionString))
{
try
{
conn.Open();
}
catch
{
//if we can't connect, then append on \SQLEXPRESS to the data source
builder.DataSource = builder.DataSource + "\\SQLEXPRESS";
}
finally
{
conn.Close();
}
}
//set the connection string back to the original database instead of the master db
builder.InitialCatalog = originalCatalog;
DbConnection temp = SqlClientFactory.Instance.CreateConnection();
temp.ConnectionString = builder.ToString();
return temp;
}
}
Once I did that I coudl run this code in my Global.asax with no issues
Database.SetInitializer(new MigrateDatabaseToLatestVersion<ResidentPortalContext, Configuration>());
using (ResidentPortalUnitOfWork temp = new ResidentPortalUnitOfWork())
{
temp.Context.Database.Initialize(true);
}

EF Code First - Recreate Database If Model Changes

I'm currently working on a project which is using EF Code First with POCOs. I have 5 POCOs that so far depends on the POCO "User".
The POCO "User" should refer to my already existing MemberShip table "aspnet_Users" (which I map it to in the OnModelCreating method of the DbContext).
The problem is that I want to take advantage of the "Recreate Database If Model changes" feature as Scott Gu shows at: http://weblogs.asp.net/scottgu/archive/2010/07/16/code-first-development-with-entity-framework-4.aspx - What the feature basically does is to recreate the database as soon as it sees any changes in my POCOs. What I want it to do is to Recreate the database but to somehow NOT delete the whole Database so that aspnet_Users is still alive. However it seems impossible as it either makes a whole new Database or replaces the current one with..
So my question is: Am I doomed to define my database tables by hand, or can I somehow merge my POCOs into my current database and still take use of the feature without wipeing it all?
As of EF Code First in CTP5, this is not possible. Code First will drop and create your database or it does not touch it at all. I think in your case, you should manually create your full database and then try to come up with an object model that matches the DB.
That said, EF team is actively working on the feature that you are looking for: altering the database instead of recreating it:
Code First Database Evolution (aka Migrations)
I was just able to do this in EF 4.1 with the following considerations:
CodeFirst
DropCreateDatabaseAlways
keeping the same connection string and database name
The database is still deleted and recreated - it has to be to for the schema to reflect your model changes -- but your data remains intact.
Here's how: you read your database into your in-memory POCO objects, and then after the POCO objects have successfully made it into memory, you then let EF drop and recreate the database. Here is an example
public class NorthwindDbContextInitializer : DropCreateDatabaseAlways<NorthindDbContext> {
/// <summary>
/// Connection from which to ead the data from, to insert into the new database.
/// Not the same connection instance as the DbContext, but may have the same connection string.
/// </summary>
DbConnection connection;
Dictionary<Tuple<PropertyInfo,Type>, System.Collections.IEnumerable> map;
public NorthwindDbContextInitializer(DbConnection connection, Dictionary<Tuple<PropertyInfo, Type>, System.Collections.IEnumerable> map = null) {
this.connection = connection;
this.map = map ?? ReadDataIntoMemory();
}
//read data into memory BEFORE database is dropped
Dictionary<Tuple<PropertyInfo, Type>, System.Collections.IEnumerable> ReadDataIntoMemory() {
Dictionary<Tuple<PropertyInfo,Type>, System.Collections.IEnumerable> map = new Dictionary<Tuple<PropertyInfo,Type>,System.Collections.IEnumerable>();
switch (connection.State) {
case System.Data.ConnectionState.Closed:
connection.Open();
break;
}
using (this.connection) {
var metaquery = from p in typeof(NorthindDbContext).GetProperties().Where(p => p.PropertyType.IsGenericType)
let elementType = p.PropertyType.GetGenericArguments()[0]
let dbsetType = typeof(DbSet<>).MakeGenericType(elementType)
where dbsetType.IsAssignableFrom(p.PropertyType)
select new Tuple<PropertyInfo, Type>(p, elementType);
foreach (var tuple in metaquery) {
map.Add(tuple, ExecuteReader(tuple));
}
this.connection.Close();
Database.Delete(this.connection);//call explicitly or else if you let the framework do this implicitly, it will complain the connection is in use.
}
return map;
}
protected override void Seed(NorthindDbContext context) {
foreach (var keyvalue in this.map) {
foreach (var obj in (System.Collections.IEnumerable)keyvalue.Value) {
PropertyInfo p = keyvalue.Key.Item1;
dynamic dbset = p.GetValue(context, null);
dbset.Add(((dynamic)obj));
}
}
context.SaveChanges();
base.Seed(context);
}
System.Collections.IEnumerable ExecuteReader(Tuple<PropertyInfo, Type> tuple) {
DbCommand cmd = this.connection.CreateCommand();
cmd.CommandText = string.Format("select * from [dbo].[{0}]", tuple.Item2.Name);
DbDataReader reader = cmd.ExecuteReader();
using (reader) {
ConstructorInfo ctor = typeof(Test.ObjectReader<>).MakeGenericType(tuple.Item2)
.GetConstructors()[0];
ParameterExpression p = Expression.Parameter(typeof(DbDataReader));
LambdaExpression newlambda = Expression.Lambda(Expression.New(ctor, p), p);
System.Collections.IEnumerable objreader = (System.Collections.IEnumerable)newlambda.Compile().DynamicInvoke(reader);
MethodCallExpression toArray = Expression.Call(typeof(Enumerable),
"ToArray",
new Type[] { tuple.Item2 },
Expression.Constant(objreader));
LambdaExpression lambda = Expression.Lambda(toArray, Expression.Parameter(typeof(IEnumerable<>).MakeGenericType(tuple.Item2)));
var array = (System.Collections.IEnumerable)lambda.Compile().DynamicInvoke(new object[] { objreader });
return array;
}
}
}
This example relies on a ObjectReader class which you can find here if you need it.
I wouldn't bother with the blog articles, read the documentation.
Finally, I would still suggest you always back up your database before running the initialization. (e.g. if the Seed method throws an exception, all your data is in memory, so you risk your data being lost once the program terminates.) A model change isn't exactly an afterthought action anyway, so be sure to back your data up.
One thing you might consider is to use a 'disconnected' foreign key. You can leave the ASPNETDB alone and just reference the user in your DB using the User key (guid). You can access the logged in user as follows:
MembershipUser currentUser = Membership.GetUser(User.Identity.Name, true /* userIsOnline */);
And then use the User's key as a FK in your DB:
Guid UserId = (Guid) currentUser.ProviderUserKey ;
This approach decouples your DB with the ASPNETDB and associated provider architecturally. However, operationally, the data will of course be loosely connected since the IDs will be in each DB. Note also there will be no referential constraints, whcih may or may not be an issue for you.

MSTest with Moq - DAL setup

I'm new to Moq, and just started on a project that's already in development. I'm responsible for setting up unit testing. There's a custom class for the DatabaseFactory that uses EnterpriseLibrary and looks like this:
public Database CreateCommonDatabase()
{
return CreateDatabaseInstance(string.Empty);
}
private static Database CreateDatabaseInstance(string foo)
{
var database = clientCode == string.Empty
? DatabaseFactory.CreateDatabase("COMMON")
: new OracleDatabase(new ClientConnections().GetConnectionString(foo)));
return database;
}
Now, here's where that gets used (ResultData is another class of the type DataSet):
public ResultData GetNotifications(string foo, string foo2, Database database)
{
var errMsg = string.Empty;
var retval = 0;
var ds = new DataSet();
var sqlClause =
#"[Some SELECT statement here that uses foo]";
DbCommand cm = database.GetSqlStringCommand(sqlClause);
cm.CommandType = CommandType.Text;
// Add Parameters
if (userSeq != string.Empty)
{
database.AddInParameter(cm, ":foo2", DbType.String, foo2);
}
try
{
ds = database.ExecuteDataSet(cm);
}
catch (Exception ex)
{
retval = -99;
errMsg = ex.Message;
}
return new ResultData(ds, retval, errMsg);
}
Now, originally, the Database wasn't passed in as a parameter, but the method was creating a new instance of the DatabaseFactory using the CreateCommonDatabase method, and using it from there. However, that leaves the class untestable because I can't keep it from actually hitting the database. So, I went with Dependency Injection, and pass the Database in.
Now, I'm stuck, because there's no way to mock Database in order to test GetNotifications. I'm wondering if I'm overly complicating things, or if I'm missing something. Am I doing this the right way, or should I be rethinking how I've got this set up?
Edit to add more info*****
I really don't want to test the database. I want the Data.Notifications class (above) to return an instance of ResultData, but that's all I really want to test. If I go a level up, to the Business layer, I have this:
public DataSet GetNotifications(string foo, string foo1, out int returnValue, out string errorMessage, Database database)
{
ResultData rd = new data.Notifications().GetNotifications(foo, foo1, database);
returnValue = rd.ResultValue;
errorMessage = rd.ErrorMessage;
return rd.DataReturned;
}
So, originally, the database wasn't passed in, it was the Data.Notifications class that created it - but then again, if I left it that way, I couldn't help but hit the database to test this Business layer object. I modified all of the code to pass the Database in (which gets created a the web's Base page), but now I'm just not certain what to do next. I thought I was one unit test away from having this resolved, but apparently, either I'm wrong or I've got a mental roadblock to the right path.
You should be able to create a mock Database object if the methods in it are virtual. If they are not, then you have a little bit of a problem.
I don't know what type "Database" is, but you have a few options.
If you own the source code to Database, I would recommend extracting an interface IDatabase, rather than dealing with a Database class type. This will eliminate some complexity and give you something extremely testable.
If you don't have access to the Database class, you can always solve this with another layer of abstraction. Many people in this case use a Repository pattern that wraps the data access layer. Generally speaking in this case, most people leave testing Respository classes to integration tests (tests without any isolation), rather than unit tests.
Here's how you'd setup your test using option #1:
[TestMethod]
public void GetNotifications_PassedNullFoo_ReturnsData()
{
//Arrange
Mock<IDatabase> mockDB = new Mock<IDatabase>();
mockDB.Setup(db => db.ExecuteDataSet()).Returns(new DataSet() ... );
//Act
FooClass target = new fooClass();
var result = target.GetNotifications(null, "Foo2", mockDB.Object);
//Assert
Assert.IsTrue(result.DataSet.Rows.Count > 0);
}
My dataset code is a little rusty, but hopefully this gives you the general idea.
Based on the code you've given, I would think you would want to talk to the database, and not a mocked version.
The reason is that your GetNotifications code contains DB-specific instructions, and you'll want those to pass validation at the DB Engine level. So just pass in a Database that is connected to your test DB instance.
If you took the testing abstraction to a higher level, where you built unit tests for the database call and a version of this test that used a mocked database, you'd still have to run integration tests, which ends up being triple the work for the same amount of code coverage. In my opinion, it's far more efficient to do an integration test at tier borders you control then to write unit tests for both sides of the contract and integration tests.