I changed my appSettings.config to no longer have connection strings as they are now all in Azure Key Vault. I was able to connect no problem, but now when I try to create the db using EF code first migrations in a new azure db using:
add-migration InitialCreate
I am getting the error:
Value cannot be null.
Parameter name: connectionString
Startup.cs
public Startup(IConfiguration configuration)
{
Configuration = configuration;
}
public IConfiguration Configuration { get; }
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
// Add functionality to inject IOptions<T>
services.AddOptions();
// Other configurations here such as for Blob and Notification hub
//
//
services.AddDbContext<ObContext>(opt =>
opt.UseSqlServer(Configuration["obdbqauser"]));
My Program.cs looks like this
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((context, config) =>
{
//TODO: Seperatre dev and pro - if (context.HostingEnvironment.IsProduction())
var buildConfig = config.Build();
//Create Managed Service Identity token provider
var tokenProvider = new AzureServiceTokenProvider();
//Create the Key Vault client
var keyVaultClient = new KeyVaultClient(
new KeyVaultClient.AuthenticationCallback(
tokenProvider.KeyVaultTokenCallback));
config.AddAzureKeyVault(
$"https://{buildConfig["VaultName"]}.vault.azure.net/",
keyVaultClient,
new DefaultKeyVaultSecretManager());
})
Here is a sample for how you can configure Key Vault as a configuration source in ASP.NET Core 2.x:
public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.ConfigureAppConfiguration((ctx, builder) =>
{
//Build the config from sources we have
var config = builder.Build();
//Add Key Vault to configuration pipeline
builder.AddAzureKeyVault(config["KeyVault:BaseUrl"]);
})
.Build();
and a configuration would be like below:
services.AddDbContext<dbContext>(async options =>
{
var keyVaultUri = new Uri("https://xxxxxxxxx.vault.azure.net/");
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
SecretBundle connectionStringSecret = await keyVaultClient.GetSecretAsync(keyVaultUri + "secrets/DBConnectionString");
options.UseSqlServer(connectionStringSecret.Value);
});
You'll need Microsoft.Extensions.Configuration.AzureKeyVault to get the configuration provider for Key Vault.
The secret naming in Key Vault will matter. For example, we will override the following connection string:
{
"ConnectionStrings": {
"DefaultConnection": "..."
}
}
You would have to create a secret named ConnectionStrings--DefaultConnection with the connection string as the value.
Then while configuring you just use Configuration["ConnectionStrings:DefaultConnection"] to get the connection string. It'll come from Key Vault if Key Vault config was added and a secret with the right name was found.
For reference , please take a look at this link.
https://entityframeworkcore.com/knowledge-base/53103236/azure-keyvault-for-dbcontext---no-database-provider-has-been-configured-for-this-dbcontext-
Hope it helps.
Related
Runtime:
I am using .NET 6 and EF Core in an Azure Function. To connect with an Azure SQL Database, I want to use AAD-Authentication, so I configured my DbContext as follows:
public class FunctionContext : DbContext {
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
SqlConnection connection = new();
var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions { ManagedIdentityClientId = Environment.GetEnvironmentVariable("userAssignedClientId") });
var token = credential.GetToken(new Azure.Core.TokenRequestContext(new[] { "https://database.windows.net/.default" }));
connection.ConnectionString = Environment.GetEnvironmentVariable("SqlConnectionString");
connection.AccessToken = token.Token;
optionsBuilder.UseSqlServer(connection);
optionsBuilder.LogTo(Console.WriteLine);
optionsBuilder.UseExceptionProcessor();
optionsBuilder.EnableSensitiveDataLogging();
}
}
The connection string "SqlConnectionString" is available as an environment variable and has the following form:
"Server=demo.database.windows.net; Database=testdb";
Migrations:
I want to update the database with every deployment. I am using Azure DevOps pipelines to deploy the application, and I have a service principal that I can use to log in. So I need to use a connection string that looks like this:
"Server=demo.database.windows.net; Authentication=Active Directory Service Principal; Encrypt=True; Database=testdb; User Id=AppId; Password=secret";
Is there a possiblity to use two different connection strings for runtime and migrations?
I tried modifiying the Factory method that Update-Database uses to create the context, but since the OnConfiguring method pasted above is called anyway, I still end up with the same connection string.
The solution I found was not to implement the OnConfiguring method, but pass the configuration directly in the Startup.cs as follows:
Startup.cs (Context at runtime)
internal class Startup : FunctionsStartup
{
public override void Configure(IFunctionsHostBuilder builder)
{
builder.Services.AddDbContext<FunctionContext>(options =>
{
SqlConnection connection = new();
var credentialOptions = new DefaultAzureCredentialOptions {
ManagedIdentityClientId = Environment.GetEnvironmentVariable("userAssignedClientId")};
var credential = new DefaultAzureCredential(credentialOptions);
var token = credential.GetToken(new Azure.Core.TokenRequestContext(new[] { "https://database.windows.net/.default" }));
connection.ConnectionString = Environment.GetEnvironmentVariable("SqlConnectionString");
connection.AccessToken = token.Token;
options.UseSqlServer(connection);
options.LogTo(Console.WriteLine);
options.UseExceptionProcessor();
options.EnableSensitiveDataLogging();
});
}
}
DesignTimeFunctionContextFactory.cs (Context at design time)
public class DesignTimeFunctionContextFactory : IDesignTimeDbContextFactory<FunctionContext>
{
public FunctionContext CreateDbContext(string[] args)
{
var optionsBuilder = new DbContextOptionsBuilder<FunctionContext>();
optionsBuilder.UseSqlServer(Environment.GetEnvironmentVariable("SqlAdminConnectionString"), options => options.EnableRetryOnFailure());
return new FunctionContext(optionsBuilder.Options);
}
}
I have a design where I have one "master" database and multiple "client" databases. When I get a request I lookup in the master database and setup the connection to the right client database.
I'm now trying to design the same in .net 5, where I setup the masterDB in StartUps ConfigureServices():
services.AddDbContext<Models.DataContext.MasterContext>(options =>
options.UseSqlServer("Name=MasterDB"));
I then on the request lookup in the MasterDB as the first thing in every controllers methods and find the connectionString for the clientDB.
But how do I then set it up at that point in time? While also not having to think about disposal of the connection, like when it's passed in using dependency injection, it's handled.
Any advice to do things slightly different are also encouraged.
Inject your MasterContext into a service that provides connection string lookups for your "client" databases (probably with caching). Then use that when resolving and configuring your "client" DbContext.
Something like this:
class ClientDatabaseService
{
MasterDbContext db;
IHttpContextAccessor context;
static Dictionary<string, string> cache = null;
public ClientDatabaseService(MasterDbContext db, IHttpContextAccessor context)
{
this.db = db;
this.context = context;
if (cache == null) RefreshCache();
}
public void RefreshCache()
{
cache = db.Clients.Select(c => new { c.ClientID, c.ConnectionString }).ToDictionary(c => c.ClientID, c => c.ConnectionString);
}
public string GetClientConnectionString()
{
var clientId = context.HttpContext.User.FindFirst("ClientID").Value;
return cache[clientId];
}
}
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddDbContext<MasterDbContext>();
services.AddHttpContextAccessor();
services.AddScoped<ClientDatabaseService>();
services.AddDbContext<ClientDbContext>((services, options) =>
{
var constrService = services.GetRequiredService<ClientDatabaseService>();
var constr = constrService.GetClientConnectionString();
options.UseSqlServer(constr, o => o.UseRelationalNulls());
});
}
I'm using Autofac for injecting dependencies in Web Api.
I set InstancePerRequest scope for EF DBContext.
Autofac Wiring up configuration:
public void Configuration(IAppBuilder app)
{
ConfigureOAuth(app);
var config = new HttpConfiguration();
var builder = new ContainerBuilder();
builder.RegisterApiControllers(Assembly.GetExecutingAssembly()).InstancePerRequest();
var asmb = typeof (TrafficDataService).Assembly;
builder.RegisterAssemblyTypes(asmb).Where(t => t.Name.EndsWith("Service")).AsImplementedInterfaces().InstancePerRequest();
builder.RegisterType<TrafficServiceGlobalContext>().As<IUnitOfWork>().InstancePerRequest();
builder.RegisterType<EMSEntities>().As<IEmsDataModel>().InstancePerRequest();
builder.RegisterType<ViolationTrafficEntities>().As<IViolationDataModel>().InstancePerRequest();
builder.RegisterType<TrafficController>().As<IHttpController>().InstancePerRequest();
builder.Register(c => HttpContext.Current.GetOwinContext().Authentication).InstancePerRequest();
builder.Register(c => app.GetDataProtectionProvider()).InstancePerRequest();
var container = builder.Build();
config.DependencyResolver = new AutofacWebApiDependencyResolver(container);
app.UseAutofacMiddleware(container);
app.UseAutofacWebApi(config);
app.UseCors(Microsoft.Owin.Cors.CorsOptions.AllowAll);
app.UseWebApi(config);
WebApiConfig.Register(config);
}
After Service gets huge number of request, and when checking Active Sessions for my DB i see 500 Active Sessions over db.
Is really this a problem?
How to implement Connection Pooling?
Any idea?
Update:
All relevant classes depends on interfaces.
As Yacoub Massad asked in comments area, here is some of relevant classes signature and constructors:
public partial class ViolationTrafficEntities : DbContext, IViolationDataModel
{
.
.
.
}
public class TrafficDataService : ITrafficDataService
{
private readonly IViolationDataModel _violationDataModel;
public TrafficDataService(IViolationDataModel violationDataModel)
{
_violationDataModel = violationDataModel;
}
}
I recently upgraded to a new version of Hangfire and I am struggeling trying to setup my webapi with autofac and Hangfire. I'm using Autofac Hangfire integration version 1.1 and Hangfire 1.4.2. I'm using Owin to host. I keep getting following error:
The requested service 'IFoo' has not been registered. To avoid this exception, either register a component to provide the service, check for service registration using IsRegistered(), or use the ResolveOptional() method to resolve an optional dependency.
Here is my owin startup configuration. All my registrations are made in the AutofacStandardModule class
public class Startup
{
public void Configuration(IAppBuilder app)
{
//we will have the firewall block all CE endpoints from the outside instead
//ConfigureOAuthTokenConsumption(app);
var storage = new SqlServerStorage("connection string");
JobStorage.Current = storage;
app.UseHangfireServer(new BackgroundJobServerOptions(),storage);
app.UseHangfireDashboard("/Hangfire",new DashboardOptions(),storage);
var builder = new ContainerBuilder();
builder.RegisterModule(new AutofacStandardModule());
var container = builder.Build();
GlobalConfiguration.Configuration.UseAutofacActivator(container);
}
}
Also, here is my web api config class. I dont see how I should be configuring Hangfire here also though..
public static class WebApiConfig
{
public static void Register(HttpConfiguration config, Autofac.Module moduleToAppend)
{
config.MapHttpAttributeRoutes();
config.EnableCors();
config.EnableSystemDiagnosticsTracing();
var builder = new ContainerBuilder();
builder.RegisterAssemblyTypes(
Assembly.GetExecutingAssembly())
.Where(t =>
!t.IsAbstract && typeof(ApiController).IsAssignableFrom(t))
.InstancePerLifetimeScope();
builder.RegisterModule(
new AutofacStandardModule());
if (moduleToAppend != null)
{
builder.RegisterModule(moduleToAppend);
}
var container = builder.Build();
config.DependencyResolver = new AutofacWebApiDependencyResolver(
container);
//Hangfire.GlobalConfiguration.Configuration.UseAutofacActivator(container);
//JobActivator.Current = new AutofacJobActivator(container);
}
}
I solved the issue, it seemed I hadn't specified clearly enough which type my job was when enqueuing.
What is did was to change
_jobClient.Enqueue(
() => _foo.Bar(fooId, fooId2));
..into..
_jobClient.Enqueue<IFoo>(x => x.Bar(fooId, fooId2));
Lets say we have the architecture model of web application where we have 1 database per 1 account. Database structure is the same for these accounts and differs only on data with in. How can i configurate a migrations in code first model.
Now I have next solution.
In the main method or in global.asax something like this:
var migration_config = new Configuration();
migration_config.TargetDatabase = new DbConnectionInfo("BlogContext");
var migrator = new DbMigrator(migration_config);
migrator.Update();
migration_config.TargetDatabase = new DbConnectionInfo("BlogContextCopy");
migrator = new DbMigrator(migration_config);
migrator.Update();
Connection strings for example in app_config file:
<connectionStrings>
<add name="BlogContext" providerName="System.Data.SqlClient" connectionString="Server=(localdb)\v11.0;Database=MigrationsDemo.BlogContext;Integrated Security=True;"/>
<add name="BlogContextCopy" providerName="System.Data.SqlClient" connectionString="Server=(localdb)\v11.0;Database=MigrationsDemo.BlogContextCopy;Integrated Security=True;"/>
</connectionStrings>
Configuration class and context:
internal sealed class Configuration : DbMigrationsConfiguration<MigrationsDemo.BlogContext>
{
public Configuration()
{
AutomaticMigrationsEnabled = true;
AutomaticMigrationDataLossAllowed = true;
}
protected override void Seed(MigrationsDemo.BlogContext context) {
}
}
public class BlogContext : DbContext {
public BlogContext() {}
public BlogContext(string connection_name) : base(connection_name) {
}
public DbSet<Blog> Blogs { get; set; }
}
In addition to your excellent answer, you can use an external config file (i.e. "clients.json") instead of hardcoding them, put all the database infos in key-value pairs into the json file and load it during startup.
Then, by iterating over the key-value pairs, you can do the initialization.
The clients.json:
{
"DatabaseA": "DatabaseAConnectionString",
"DatabaseB": "DatabaseBConnectionString",
"DatabaseC": "DatabaseCConnectionString",
...
}
Provide a method to handle the migrations:
public static void MigrateDatabases(IDictionary<string,string> databaseConfigs)
{
foreach (var db in databaseConfigs)
{
var config = new Configuration
{
TargetDatabase = new DbConnectionInfo(db.Value, "System.Data.SqlClient")
};
var migrator = new DbMigrator(config);
migrator.Update();
}
}
Then during startup, (I use OWIN, so it's in my Startup.cs, could also be global.asax.cs):
string json;
var path = HttpRuntime.AppDomainAppPath;
using (var reader = new StreamReader(path + #"Config\clients.json"))
{
json = reader.ReadToEnd();
}
var databases = JsonConvert.DeserializeObject<IDictionary<string, string>>(json);
MigrateDatabases(databases);
Works like a charm for me :)
See the page on automatic migrations during application startup.
If you use this method to apply your migrations, you can use any connection string (or whatever method you have to identify exactly which database to connect to) and upon connection, the migration will be performed.