Runtime:
I am using .NET 6 and EF Core in an Azure Function. To connect with an Azure SQL Database, I want to use AAD-Authentication, so I configured my DbContext as follows:
public class FunctionContext : DbContext {
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
SqlConnection connection = new();
var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions { ManagedIdentityClientId = Environment.GetEnvironmentVariable("userAssignedClientId") });
var token = credential.GetToken(new Azure.Core.TokenRequestContext(new[] { "https://database.windows.net/.default" }));
connection.ConnectionString = Environment.GetEnvironmentVariable("SqlConnectionString");
connection.AccessToken = token.Token;
optionsBuilder.UseSqlServer(connection);
optionsBuilder.LogTo(Console.WriteLine);
optionsBuilder.UseExceptionProcessor();
optionsBuilder.EnableSensitiveDataLogging();
}
}
The connection string "SqlConnectionString" is available as an environment variable and has the following form:
"Server=demo.database.windows.net; Database=testdb";
Migrations:
I want to update the database with every deployment. I am using Azure DevOps pipelines to deploy the application, and I have a service principal that I can use to log in. So I need to use a connection string that looks like this:
"Server=demo.database.windows.net; Authentication=Active Directory Service Principal; Encrypt=True; Database=testdb; User Id=AppId; Password=secret";
Is there a possiblity to use two different connection strings for runtime and migrations?
I tried modifiying the Factory method that Update-Database uses to create the context, but since the OnConfiguring method pasted above is called anyway, I still end up with the same connection string.
The solution I found was not to implement the OnConfiguring method, but pass the configuration directly in the Startup.cs as follows:
Startup.cs (Context at runtime)
internal class Startup : FunctionsStartup
{
public override void Configure(IFunctionsHostBuilder builder)
{
builder.Services.AddDbContext<FunctionContext>(options =>
{
SqlConnection connection = new();
var credentialOptions = new DefaultAzureCredentialOptions {
ManagedIdentityClientId = Environment.GetEnvironmentVariable("userAssignedClientId")};
var credential = new DefaultAzureCredential(credentialOptions);
var token = credential.GetToken(new Azure.Core.TokenRequestContext(new[] { "https://database.windows.net/.default" }));
connection.ConnectionString = Environment.GetEnvironmentVariable("SqlConnectionString");
connection.AccessToken = token.Token;
options.UseSqlServer(connection);
options.LogTo(Console.WriteLine);
options.UseExceptionProcessor();
options.EnableSensitiveDataLogging();
});
}
}
DesignTimeFunctionContextFactory.cs (Context at design time)
public class DesignTimeFunctionContextFactory : IDesignTimeDbContextFactory<FunctionContext>
{
public FunctionContext CreateDbContext(string[] args)
{
var optionsBuilder = new DbContextOptionsBuilder<FunctionContext>();
optionsBuilder.UseSqlServer(Environment.GetEnvironmentVariable("SqlAdminConnectionString"), options => options.EnableRetryOnFailure());
return new FunctionContext(optionsBuilder.Options);
}
}
Related
I'm having problems with seeding my inmemory database with test-data in nUnit integration tests.
I've created a custom WebApplicationFactory so i could use an inmemory database:
public class SiriusWebApplicationFactory : WebApplicationFactory<Startup>
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureTestServices(services =>
{
var descriptor = services.SingleOrDefault(d => d.ServiceType == typeof(IAuthenticationSchemeProvider));
services.Remove(descriptor);
CleanupDatabaseRegistrations<SiriusContext>(services);
services.AddDbContext<SiriusContext>(options => options.UseInMemoryDatabase($"Testdb-{Guid.NewGuid()}"));
// Build the service provider.
var sp = services.BuildServiceProvider();
// Create a scope to obtain a reference to the database contexts
using var scope = sp.CreateScope();
var scopedServices = scope.ServiceProvider;
var context = scopedServices.GetRequiredService<SiriusContext>();
// Ensure the database is created.
context.Database.EnsureCreated();
});
}
private void CleanupDatabaseRegistrations<TDbContext>(IServiceCollection services) where TDbContext : DbContext
{
var descriptor = services.SingleOrDefault(d => d.ServiceType == typeof(DbContextOptions<TDbContext>));
if (descriptor != null)
{
services.Remove(descriptor);
}
descriptor = services.FirstOrDefault(d => d.ServiceType == typeof(DbContextOptions));
if (descriptor != null)
{
services.Remove(descriptor);
}
descriptor = services.SingleOrDefault(d => d.ServiceType == typeof(TDbContext));
if (descriptor != null)
{
services.Remove(descriptor);
}
}
}
After this, i've created a base class which is being used by all my integration test classes.
This base class implements the nUnit OneTimeSetup method where i create the HttpClient for testing my controllers:
protected HttpClient HttpClient;
protected SiriusContext DbContext;
[OneTimeSetUp]
public void OneTimeSetUp()
{
_webApplicationFactory = new SiriusWebApplicationFactory();
HttpClient = _webApplicationFactory.CreateClient();
DbContext = _webApplicationFactory.Services.GetService<SiriusContext>();
DoReseed();
}
As you can see i'm exposing my DbContext so that my test classes can seed the test data.
The DoReseed method is just an emppty method used by my integration test classes to provide test data.
protected virtual void DoReseed() { }
The implementation inside one of my testclasses looks like this:
public class AdminAanbestedingApplicatieControllerTests : BaseIntegrationTest
{
[Test]
public async Task AanbestedingApplicaties_GetAll_ShouldReturnData()
{
var result = await HttpClient.GetFromApi_AssertSuccess<List<AanbestedingApplicatie>>(RouteConstants.Lijsten.AdminAanbestedingApplicaties.GetAll);
Assert.IsTrue(result.Count > 1);
}
protected override void DoReseed()
{
DbContext.AanbestedingApplicaties.Add(AanbestedingApplicatiesObjectMother.AanbestedingApplicatie_Actief);
DbContext.AanbestedingApplicaties.Add(AanbestedingApplicatiesObjectMother.AanbestedingApplicatie_InActief);
DbContext.AanbestedingApplicaties.Add(AanbestedingApplicatiesObjectMother.AanbestedingApplicatie_ToDelete);
DbContext.AanbestedingApplicaties.Add(AanbestedingApplicatiesObjectMother.AanbestedingApplicatie_ToUpdate);
DbContext.SaveChanges();
}
}
When i execute my test, the seeding gets executed but the repository cannot see the seeded data?
The DbContext inside my repo does not contain my seeded data...
Anyone has any idea why the DbContext inside my repository does not contain the seeded test data?
The repo looks like this:
public IEnumerable<AanbestedingApplicatie> GetAll()
{
return SiriusContext.AanbestedingApplicaties.OrderBy(l => l.Naam).ProjectTo<AanbestedingApplicatie>(Mapper.ConfigurationProvider)
.ToList();
}
The Context has been injected inside the constructor...
I have a design where I have one "master" database and multiple "client" databases. When I get a request I lookup in the master database and setup the connection to the right client database.
I'm now trying to design the same in .net 5, where I setup the masterDB in StartUps ConfigureServices():
services.AddDbContext<Models.DataContext.MasterContext>(options =>
options.UseSqlServer("Name=MasterDB"));
I then on the request lookup in the MasterDB as the first thing in every controllers methods and find the connectionString for the clientDB.
But how do I then set it up at that point in time? While also not having to think about disposal of the connection, like when it's passed in using dependency injection, it's handled.
Any advice to do things slightly different are also encouraged.
Inject your MasterContext into a service that provides connection string lookups for your "client" databases (probably with caching). Then use that when resolving and configuring your "client" DbContext.
Something like this:
class ClientDatabaseService
{
MasterDbContext db;
IHttpContextAccessor context;
static Dictionary<string, string> cache = null;
public ClientDatabaseService(MasterDbContext db, IHttpContextAccessor context)
{
this.db = db;
this.context = context;
if (cache == null) RefreshCache();
}
public void RefreshCache()
{
cache = db.Clients.Select(c => new { c.ClientID, c.ConnectionString }).ToDictionary(c => c.ClientID, c => c.ConnectionString);
}
public string GetClientConnectionString()
{
var clientId = context.HttpContext.User.FindFirst("ClientID").Value;
return cache[clientId];
}
}
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddDbContext<MasterDbContext>();
services.AddHttpContextAccessor();
services.AddScoped<ClientDatabaseService>();
services.AddDbContext<ClientDbContext>((services, options) =>
{
var constrService = services.GetRequiredService<ClientDatabaseService>();
var constr = constrService.GetClientConnectionString();
options.UseSqlServer(constr, o => o.UseRelationalNulls());
});
}
Is there any code to perform automatic migration in Entity Framework core code first in asp.net core project?
I do it simply in MVC4/5 by adding
Database.SetInitializer(new MigrateDatabaseToLatestVersion<AppDbContext, MyProject.Migrations.Configuration>());
public Configuration() {
AutomaticMigrationsEnabled = true;
}
This saves time when entities changed
You can call context.Database.Migrate()in your Startup.cs
eg:
using (var context = new MyContext(...))
{
context.Database.Migrate();
}
EF core doesn't support automatic migrations.So you have to do it manually.
From the perspective of automatic migrations as a feature, we are not
planning to implement it in EF Core as experience has showed code-base
migrations to be a more manageable approach.
You can read full story here : Not to implement Automatic Migrations
This is the way they do it in IdentityServer4 http://identityserver.io
public void ConfigureServices(IServiceCollection services)
{
var connectionString = Configuration.GetConnectionString("DefaultConnection");
var migrationsAssembly = typeof(Startup).GetTypeInfo().Assembly.GetName().Name;
// Add framework services.
services.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlServer(connectionString));
...
}
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
// this will do the initial DB population
InitializeDatabase(app);
}
private void InitializeDatabase(IApplicationBuilder app)
{
using (var scope = app.ApplicationServices.GetService<IServiceScopeFactory>().CreateScope())
{
scope.ServiceProvider.GetRequiredService<ApplicationDbContext>().Database.Migrate();
scope.ServiceProvider.GetRequiredService<PersistedGrantDbContext>().Database.Migrate();
...
}
}
Automatic migrations is not supported in EF Core. Migration it is necessary to create hands. To automatically apply all existing handmade migrations need to add the following code in the class Program:
public class Program
{
public static void Main(string[] args)
{
var host = CreateWebHostBuilder(args).Build();
using (var scope = host.Services.CreateScope())
{
var services = scope.ServiceProvider;
try
{
var context = services.GetRequiredService<MyDbContext>();
context.Database.Migrate(); // apply all migrations
SeedData.Initialize(services); // Insert default data
}
catch (Exception ex)
{
var logger = services.GetRequiredService<ILogger<Program>>();
logger.LogError(ex, "An error occurred seeding the DB.");
}
}
host.Run();
}
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>();
}
Following Microsoft's documentation
https://learn.microsoft.com/en-us/aspnet/core/data/ef-mvc/intro
If you are using dependency injection, first, you need to setup a static class Data/DbInitializer.cs and add the following code:
public static class DbInitializer
{
public static void Initialize(ApplicationDbContext context)
{
context.Database.Migrate();
// Add Seed Data...
}
}
Notice, this is also where you can add seed data.
Next, in your Program.cs file, add the following code
public static void Main(string[] args)
{
var host = BuildWebHost(args);
using (var scope = host.Services.CreateScope())
{
var services = scope.ServiceProvider;
try
{
var environment = services.GetRequiredService<IHostingEnvironment>();
if (!environment.IsDevelopment())
{
var context = services.GetRequiredService<ApplicationDbContext>();
DbInitializer.Initialize(context);
}
}
catch (Exception ex)
{
var logger = services.GetRequiredService<ILogger<Program>>();
logger.LogError(ex, "An error occurred while seeding the database.");
}
}
host.Run();
}
In my case, I'm checking the environment to make sure I'm in development so I can control the migrations/updates. However, in production, I want them to be automatic for continuous integration. As others have mentioned, this is probably not best practices but on small projects it works great.
My working automigration code Asp Net Core 2.0.7.
// startup.cs
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
// configure app
SeedData.Initialize(app.ApplicationServices);
}
// dbInitializer.cs
public static class SeedData
{
public static void Initialize(IServiceProvider serviceProvider)
{
using (var serviceScope = serviceProvider.CreateScope())
{
var context = serviceScope.ServiceProvider.GetService<ApplicationDbContext>();
// auto migration
context.Database.Migrate();
// Seed the database.
InitializeUserAndRoles(context);
}
}
private static void InitializeUserAndRoles(ApplicationDbContext context)
{
// init user and roles
}
}
You can call Database.Migrate() in db context constructor.
If the model changes a lot and you manage a medium - large team, migrations leads more problems than solution at least in development phase.
I published a nuget package with automatic migration for .net core, EFCore.AutomaticMigrations - https://www.nuget.org/packages/EFCore.AutomaticMigrations/, so manual migration not needed anymore.
You can call directly in Program class, like bellow:
public static void Main(string[] args)
{
var host = CreateWebHostBuilder(args);
using (var scope = host.Services.CreateScope())
{
var services = scope.ServiceProvider;
var loggerFactory = services.GetRequiredService<ILoggerFactory>();
var logger = loggerFactory.CreateLogger<Program>();
try
{
var environment = services.GetRequiredService<IWebHostEnvironment>();
if (environment.IsDevelopment())
{
var context = services.GetRequiredService<ApplicationContext>();
MigrateDatabaseToLatestVersion.ExecuteAsync(context).Wait();
}
}
catch (Exception ex)
{
logger.LogError(ex, "An error occurred creating/updating the DB.");
}
}
host.Run();
}
Frank Odoom's answer works even 4 years later in .net 5, but it is not the intended context to call the migration at runtime... And, it appears it never was because it requires us to mock the DbContext with DbContextOptions whos documentation explicitly states:
"The options to be used by a DbContext. You normally override OnConfiguring(DbContextOptionsBuilder) or use a DbContextOptionsBuilder to create instances of this class and it is not designed to be directly constructed in your application code."
Here is my suggestion:
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
// database provider is configured before runtime migration update is applied e.g:
optionsBuilder.UseSqlServer(ConnectionString);
Database.Migrate();
}
Edit:
My suggestion is actually horrible if you are using multiple DBContexts in the same project... It would migrate the database multiple times. Which would most likely not break anything, but it would slow startup considerably.
my best advice is not to use the automatic migration.It is always better to add migrations manually and also avoid bulk migration and stick to best practice for using manual migration
automatic migration is not a magic tool and there will be several occasions where you might want to add some addition changes to the migration. You only accomplish by using manual migration.
To enable migration, type "enable-migrations" in the package manager console
This way you will have full control of upgrading or downgrading your database and also easy to track migrations.
Just three simple steps in package manager console.
1) add-migrations [some name for your migration]
2) migrations is generated for the changes, you review them and also can
make changes to it
3) update-database your migration is complete now.
handling migration is less painful!
Lets say we have the architecture model of web application where we have 1 database per 1 account. Database structure is the same for these accounts and differs only on data with in. How can i configurate a migrations in code first model.
Now I have next solution.
In the main method or in global.asax something like this:
var migration_config = new Configuration();
migration_config.TargetDatabase = new DbConnectionInfo("BlogContext");
var migrator = new DbMigrator(migration_config);
migrator.Update();
migration_config.TargetDatabase = new DbConnectionInfo("BlogContextCopy");
migrator = new DbMigrator(migration_config);
migrator.Update();
Connection strings for example in app_config file:
<connectionStrings>
<add name="BlogContext" providerName="System.Data.SqlClient" connectionString="Server=(localdb)\v11.0;Database=MigrationsDemo.BlogContext;Integrated Security=True;"/>
<add name="BlogContextCopy" providerName="System.Data.SqlClient" connectionString="Server=(localdb)\v11.0;Database=MigrationsDemo.BlogContextCopy;Integrated Security=True;"/>
</connectionStrings>
Configuration class and context:
internal sealed class Configuration : DbMigrationsConfiguration<MigrationsDemo.BlogContext>
{
public Configuration()
{
AutomaticMigrationsEnabled = true;
AutomaticMigrationDataLossAllowed = true;
}
protected override void Seed(MigrationsDemo.BlogContext context) {
}
}
public class BlogContext : DbContext {
public BlogContext() {}
public BlogContext(string connection_name) : base(connection_name) {
}
public DbSet<Blog> Blogs { get; set; }
}
In addition to your excellent answer, you can use an external config file (i.e. "clients.json") instead of hardcoding them, put all the database infos in key-value pairs into the json file and load it during startup.
Then, by iterating over the key-value pairs, you can do the initialization.
The clients.json:
{
"DatabaseA": "DatabaseAConnectionString",
"DatabaseB": "DatabaseBConnectionString",
"DatabaseC": "DatabaseCConnectionString",
...
}
Provide a method to handle the migrations:
public static void MigrateDatabases(IDictionary<string,string> databaseConfigs)
{
foreach (var db in databaseConfigs)
{
var config = new Configuration
{
TargetDatabase = new DbConnectionInfo(db.Value, "System.Data.SqlClient")
};
var migrator = new DbMigrator(config);
migrator.Update();
}
}
Then during startup, (I use OWIN, so it's in my Startup.cs, could also be global.asax.cs):
string json;
var path = HttpRuntime.AppDomainAppPath;
using (var reader = new StreamReader(path + #"Config\clients.json"))
{
json = reader.ReadToEnd();
}
var databases = JsonConvert.DeserializeObject<IDictionary<string, string>>(json);
MigrateDatabases(databases);
Works like a charm for me :)
See the page on automatic migrations during application startup.
If you use this method to apply your migrations, you can use any connection string (or whatever method you have to identify exactly which database to connect to) and upon connection, the migration will be performed.
I'm trying to setup some unit tests using EntityFramework 5, SQL Server Compact 4 and Xunit.
I'm using different context instances because I'm testing a ASP MVC app and I need to test the behavior of some update operations over detached entities.
[Fact, AutoRollback]
public void TestConnection()
{
using (var connection = this.GetDbConnection())
{
using (var context = new MyContext(connection, false))
{
// Do database stuff
}
using (var context = new MyContext(connection, false))
{
// Do database stuff
}
}
}
public DbConnection GetDbConnection()
{
string dataSource = "|DataDirectory|\\MyDb.sdf";
var sqlBuilder = new SqlCeConnectionStringBuilder();
sqlBuilder.DataSource = dataSource;
return new SqlCeConnection(sqlBuilder.ToString());
}
This gives me the following error:
System.Data.EntityException : The underlying provider failed on Open.
System.InvalidOperationException : The connection object can not be enlisted in transaction scope.
I know I can't open multiple DbContext instances inside a TransactionScope (that is probably what Xunit does when you put a FallbackAttribute in your method), so that's why I'm creating the connection beforehand.
If I try to open the connection myself, it still does not work:
using (var connection = this.GetDbConnection())
{
connection.Open();
using (var context = new MyContext(connection, false))
{
I get the following exception:
System.ArgumentException : EntityConnection can only be constructed with a closed DbConnection.
Does any one know how to solve that issue?
EDIT
The test classes that deal with the Db extend a "DomainFactsBase" where the database is initialized as the following:
public DomainFactsBase()
{
Database.SetInitializer(new DropCreateDatabaseIfModelChanges<MyContext>());
using (var context = new MyContext(GetDbConnection(), true))
context.Database.Initialize(false);
}
EDIT
I can sucessfully run tests with autorollback when I create only one context instance. This was accomplished following the instructions in this article. I have a extension method:
public static void OpenConnection(this DbContext context)
{
((IObjectContextAdapter)context).ObjectContext.Connection.Open();
}
And I call it right after creating the context in the tests:
[Fact, AutoRollback]
public void SomeFact()
{
using (var context = new MyContext())
{
context.OpenConnection();
// Do stuff
}
}
That work with no problems. They arise when I try to open the context more than once in the same fact (with AutoRollback enabled), as I examplified in the beginning.
Initialize the database outside of the test. You can do this inside of the test class's constructor.
public MyTestClass()
{
using (var db = new MyContext(GetDbConnection(), true))
{
db.Database.Initialize(false);
}
}